sed grouping now working in perl
by amateurscripter from LinuxQuestions.org on (#4T2QE)
Hello,
I'm trying to run a shell tool(sed) in perl but it's not working as it does when I run it from a shell command prompt.
I have the following perl loop, that's going through each message and only caring about 35=D msg types. I then try to manipulate the msg(string) using a shell command($msg) but for some reason the grouping section does not work. I only care about the timestamp in the beginning of the message, the values in tag 52 and tag 11. I tried to extract those values with the sed tool but it's not working. Is grouping now allowed in perl? What am I doing wrong? Thanks ahead of time for looking at this.
while(<INFILE>) {
@orders=grep /35=D/, $_;
foreach $line (@orders){
chomp($line);
print "$line\n";
$msg=`echo "$line"|tr "^A" "|" |tr " " "|"|sed 's/^\([^|]*\).*|52=\([^|]*\)|.*|11=\([^|]*\).*/\1 \2 \3/g'|tr " " "|"|sed 's/|.*\-[0-3][0-9]:/|/g'|cut -d':' -f2-5 `;
print "$msg\n";
}
Here are some sample data that the script would process:
07:10:38.070 IN: 8=FIX.4.29=18535=D34=25849=CLIENT52=20191023-11:10:38.06956=SERVER57=INET1=5AO0510611=3963415=USD21=138=50040=244=3.9647=A54=155=NCDA60=20191023-11:10:38.069111=0832=K9281=VSLO10=168
07:10:41.336 IN: 8=FIX.4.29=18535=D34=25949=CLIENT52=20191023-11:10:41.33656=SERVER57=INET1=5AO0510611=3996715=USD21=138=50040=244=3.8947=A54=255=NCDA60=20191023-11:10:41.336111=0832=K9281=VSLO10=154
07:11:22.926 IN: 8=FIX.4.29=18435=D34=26149=CLIENT52=20191023-11:11:22.92656=SERVER57=INET1=5AO0510611=408915=USD21=138=20040=244=3.947=A54=255=NCDA60=20191023-11:11:22.926111=0832=K9281=VSLO10=079
07:11:36.685 IN: 8=FIX.4.29=18735=D34=26249=CLIENT52=20191023-11:11:36.68456=SERVER57=INET1=5AO0515511=4065415=USD21=138=200040=244=14.1647=A54=155=BACD60=2019102 3-11:11:36.684111=0832=K9281=VSLO10=000
07:11:42.250 IN: 8=FIX.4.29=18635=D34=26449=CLIENT52=20191023-11:11:42.25056=SERVER57=INET1=5AO0515511=41342315=USD21=138=30040=244=14.1747=A54=155=BACD60=2019102 3-11:11:42.250111=0832=K9281=VSLO10=174
The output of the "print $msg" looks like this for the above line:
10:38.070|10:38.069|111=0|832=K|9281=VSLO|10=168|
10:41.336|10:41.336|111=0|832=K|9281=VSLO|10=154|
11:22.926|11:22.926|111=0|832=K|9281=VSLO|10=079|
11:36.685|11:36.684|111=0|832=K|9281=VSLO|10=000|
11:42.250|11:42.250|111=0|832=K|9281=VSLO|10=174|
Each FIX tag (ie: 9=185, 35=D, and etc) with their values is delimited with a caret(^A) in the above raw FIX log.
When I run the same shell command from shell prompt, I see the following output which is what I want to see:
myserver ~ $ cat fixmsg.log|tr "^A" "|" |tr " " "|"|sed 's/^\([^|]*\).*|52=\([^|]*\)|.*|11=\([^|]*\).*/\1 \2 \3/g'|tr " " "|"|sed 's/|.*\-[0-3][0-9]:/|/g'|cut -d':' -f2-5
10:38.070|10:38.069|39634
10:41.336|10:41.336|39967
11:22.926|11:22.926|4089
...
...
I think it's something with the caret but not sure. Maybe when I grep 35=D, the caret is somehow lost?
Can someone help me with this, I've been searching the web but cannot find a solution.
Thanks in advance


I'm trying to run a shell tool(sed) in perl but it's not working as it does when I run it from a shell command prompt.
I have the following perl loop, that's going through each message and only caring about 35=D msg types. I then try to manipulate the msg(string) using a shell command($msg) but for some reason the grouping section does not work. I only care about the timestamp in the beginning of the message, the values in tag 52 and tag 11. I tried to extract those values with the sed tool but it's not working. Is grouping now allowed in perl? What am I doing wrong? Thanks ahead of time for looking at this.
while(<INFILE>) {
@orders=grep /35=D/, $_;
foreach $line (@orders){
chomp($line);
print "$line\n";
$msg=`echo "$line"|tr "^A" "|" |tr " " "|"|sed 's/^\([^|]*\).*|52=\([^|]*\)|.*|11=\([^|]*\).*/\1 \2 \3/g'|tr " " "|"|sed 's/|.*\-[0-3][0-9]:/|/g'|cut -d':' -f2-5 `;
print "$msg\n";
}
Here are some sample data that the script would process:
07:10:38.070 IN: 8=FIX.4.29=18535=D34=25849=CLIENT52=20191023-11:10:38.06956=SERVER57=INET1=5AO0510611=3963415=USD21=138=50040=244=3.9647=A54=155=NCDA60=20191023-11:10:38.069111=0832=K9281=VSLO10=168
07:10:41.336 IN: 8=FIX.4.29=18535=D34=25949=CLIENT52=20191023-11:10:41.33656=SERVER57=INET1=5AO0510611=3996715=USD21=138=50040=244=3.8947=A54=255=NCDA60=20191023-11:10:41.336111=0832=K9281=VSLO10=154
07:11:22.926 IN: 8=FIX.4.29=18435=D34=26149=CLIENT52=20191023-11:11:22.92656=SERVER57=INET1=5AO0510611=408915=USD21=138=20040=244=3.947=A54=255=NCDA60=20191023-11:11:22.926111=0832=K9281=VSLO10=079
07:11:36.685 IN: 8=FIX.4.29=18735=D34=26249=CLIENT52=20191023-11:11:36.68456=SERVER57=INET1=5AO0515511=4065415=USD21=138=200040=244=14.1647=A54=155=BACD60=2019102 3-11:11:36.684111=0832=K9281=VSLO10=000
07:11:42.250 IN: 8=FIX.4.29=18635=D34=26449=CLIENT52=20191023-11:11:42.25056=SERVER57=INET1=5AO0515511=41342315=USD21=138=30040=244=14.1747=A54=155=BACD60=2019102 3-11:11:42.250111=0832=K9281=VSLO10=174
The output of the "print $msg" looks like this for the above line:
10:38.070|10:38.069|111=0|832=K|9281=VSLO|10=168|
10:41.336|10:41.336|111=0|832=K|9281=VSLO|10=154|
11:22.926|11:22.926|111=0|832=K|9281=VSLO|10=079|
11:36.685|11:36.684|111=0|832=K|9281=VSLO|10=000|
11:42.250|11:42.250|111=0|832=K|9281=VSLO|10=174|
Each FIX tag (ie: 9=185, 35=D, and etc) with their values is delimited with a caret(^A) in the above raw FIX log.
When I run the same shell command from shell prompt, I see the following output which is what I want to see:
myserver ~ $ cat fixmsg.log|tr "^A" "|" |tr " " "|"|sed 's/^\([^|]*\).*|52=\([^|]*\)|.*|11=\([^|]*\).*/\1 \2 \3/g'|tr " " "|"|sed 's/|.*\-[0-3][0-9]:/|/g'|cut -d':' -f2-5
10:38.070|10:38.069|39634
10:41.336|10:41.336|39967
11:22.926|11:22.926|4089
...
...
I think it's something with the caret but not sure. Maybe when I grep 35=D, the caret is somehow lost?
Can someone help me with this, I've been searching the web but cannot find a solution.
Thanks in advance