bash race condition issue with tee and grep
by ychaouche from LinuxQuestions.org on (#6ERBB)
Dear LQ,
The intention
I'd like to write a command that validates online openssl certificates,
plus showing basic information contained in them,
like the notbefore and notafter dates,
and the domain names it was delivered for.
Lets start with validation,
then see how to show the relevant information.
To work on online certificates I use the openssl s_client command,
so I started with this:
Code:11:09:22 ~ -1- $ openssl s_client -CApath /etc/ssl/certs/ -connect google.com:443 </dev/null | grep Verify
depth=3 C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
verify return:1
depth=2 C = US, O = Google Trust Services LLC, CN = GTS Root R1
verify return:1
depth=1 C = US, O = Google Trust Services LLC, CN = GTS CA 1C3
verify return:1
depth=0 CN = *.google.com
verify return:1
DONE
Verify return code: 0 (ok)
11:09:27 ~ -1- $So far,
so good.
The output of s_client is piped to grep
to only extract the final validation code
Code: Verify return code: 0 (ok)
previous data was printed to stderr and.
But in reality,
stdout had all the necessary information for openssl to also show the dates and dns,
which was the second operation I wanted to do.
The Idea
The output of s_client is already piped to grep,
but I'd like to also process it with the openssl x509 command
to print the desired information.
So I had the idea to use tee for this.
Code:command | tee >(command1) >(command2> /dev/null
The Execution
because google certificate is used for so many domain names,
I'll use a different domain here.
Code:11:15:39 ~ -1- $ openssl s_client -CApath /etc/ssl/certs/ -connect radioalgerie.dz:443 </dev/null | tee >(grep Verify) >(openssl x509 -text -noout | grep -E '(Before|After|DNS)') >/dev/null
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
verify return:1
depth=0 CN = *.radioalgerie.dz
verify return:1
DONE
Verify return code: 0 (ok)
11:15:56 ~ -1- $ Not Before: Nov 23 00:00:00 2022 GMT
Not After : Nov 24 23:59:59 2023 GMT
DNS:*.radioalgerie.dz, DNS:radioalgerie.dz
11:16:00 ~ -1- $
Sorting out the output
So what happened here is that the first >(grep Verify) part was executed well,
Code:...
DONE
Verify return code: 0 (ok)
Then bash printed the prompt and the result of the next command >(openssl x509 -text -noout | grep -E '(Before|After|DNS)')
Code:11:15:56 ~ -1- $ Not Before: Nov 23 00:00:00 2022 GMT
Not After : Nov 24 23:59:59 2023 GMT
DNS:*.radioalgerie.dz, DNS:radioalgerie.dzAt first I thought the PS1 prompt was missing
and something somewhere was waiting for an input
b/c as soon I hit return
a new prompt was printed,
but in reality I think there was a race condition,
the prompt was printed before the second command ended,
as you can see in this line:
Code:11:15:56 ~ -1- $ Not Before: Nov 23 00:00:00 2022 GMTThere wasn't anything waiting for my input.
The prompt was already printed,
and I was fooled to think otherwise by the output of the second command.
Code:11:15:56 ~ -1- $ Not Before: Nov 23 00:00:00 2022 GMT
Not After : Nov 24 23:59:59 2023 GMT
DNS:*.radioalgerie.dz, DNS:radioalgerie.dz
11:16:00 ~ -1- $
Desired output
After a couple tries,
I eventually got lucky and got everything printed in order
Code:
11:16:11 ~ -1- $ openssl s_client -CApath /etc/ssl/certs/ -connect radioalgerie.dz:443 </dev/null | tee >(grep Verify) >(openssl x509 -text -noout | grep -E '(Before|After|DNS)') >/dev/null
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
verify return:1
depth=0 CN = *.radioalgerie.dz
verify return:1
DONE
Verify return code: 0 (ok)
Not Before: Nov 23 00:00:00 2022 GMT
Not After : Nov 24 23:59:59 2023 GMT
DNS:*.radioalgerie.dz, DNS:radioalgerie.dz
11:16:12 ~ -1- $So my question is:
how can I avoid this race condition
and be more deterministic in the output of each command?
The intention
I'd like to write a command that validates online openssl certificates,
plus showing basic information contained in them,
like the notbefore and notafter dates,
and the domain names it was delivered for.
Lets start with validation,
then see how to show the relevant information.
To work on online certificates I use the openssl s_client command,
so I started with this:
Code:11:09:22 ~ -1- $ openssl s_client -CApath /etc/ssl/certs/ -connect google.com:443 </dev/null | grep Verify
depth=3 C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
verify return:1
depth=2 C = US, O = Google Trust Services LLC, CN = GTS Root R1
verify return:1
depth=1 C = US, O = Google Trust Services LLC, CN = GTS CA 1C3
verify return:1
depth=0 CN = *.google.com
verify return:1
DONE
Verify return code: 0 (ok)
11:09:27 ~ -1- $So far,
so good.
The output of s_client is piped to grep
to only extract the final validation code
Code: Verify return code: 0 (ok)
previous data was printed to stderr and.
But in reality,
stdout had all the necessary information for openssl to also show the dates and dns,
which was the second operation I wanted to do.
The Idea
The output of s_client is already piped to grep,
but I'd like to also process it with the openssl x509 command
to print the desired information.
So I had the idea to use tee for this.
Code:command | tee >(command1) >(command2> /dev/null
The Execution
because google certificate is used for so many domain names,
I'll use a different domain here.
Code:11:15:39 ~ -1- $ openssl s_client -CApath /etc/ssl/certs/ -connect radioalgerie.dz:443 </dev/null | tee >(grep Verify) >(openssl x509 -text -noout | grep -E '(Before|After|DNS)') >/dev/null
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
verify return:1
depth=0 CN = *.radioalgerie.dz
verify return:1
DONE
Verify return code: 0 (ok)
11:15:56 ~ -1- $ Not Before: Nov 23 00:00:00 2022 GMT
Not After : Nov 24 23:59:59 2023 GMT
DNS:*.radioalgerie.dz, DNS:radioalgerie.dz
11:16:00 ~ -1- $
Sorting out the output
So what happened here is that the first >(grep Verify) part was executed well,
Code:...
DONE
Verify return code: 0 (ok)
Then bash printed the prompt and the result of the next command >(openssl x509 -text -noout | grep -E '(Before|After|DNS)')
Code:11:15:56 ~ -1- $ Not Before: Nov 23 00:00:00 2022 GMT
Not After : Nov 24 23:59:59 2023 GMT
DNS:*.radioalgerie.dz, DNS:radioalgerie.dzAt first I thought the PS1 prompt was missing
and something somewhere was waiting for an input
b/c as soon I hit return
a new prompt was printed,
but in reality I think there was a race condition,
the prompt was printed before the second command ended,
as you can see in this line:
Code:11:15:56 ~ -1- $ Not Before: Nov 23 00:00:00 2022 GMTThere wasn't anything waiting for my input.
The prompt was already printed,
and I was fooled to think otherwise by the output of the second command.
Code:11:15:56 ~ -1- $ Not Before: Nov 23 00:00:00 2022 GMT
Not After : Nov 24 23:59:59 2023 GMT
DNS:*.radioalgerie.dz, DNS:radioalgerie.dz
11:16:00 ~ -1- $
Desired output
After a couple tries,
I eventually got lucky and got everything printed in order
Code:
11:16:11 ~ -1- $ openssl s_client -CApath /etc/ssl/certs/ -connect radioalgerie.dz:443 </dev/null | tee >(grep Verify) >(openssl x509 -text -noout | grep -E '(Before|After|DNS)') >/dev/null
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = GB, ST = Greater Manchester, L = Salford, O = Sectigo Limited, CN = Sectigo RSA Domain Validation Secure Server CA
verify return:1
depth=0 CN = *.radioalgerie.dz
verify return:1
DONE
Verify return code: 0 (ok)
Not Before: Nov 23 00:00:00 2022 GMT
Not After : Nov 24 23:59:59 2023 GMT
DNS:*.radioalgerie.dz, DNS:radioalgerie.dz
11:16:12 ~ -1- $So my question is:
how can I avoid this race condition
and be more deterministic in the output of each command?