Code Comments
Programming Forum and web based access to our favorite programming groups.
matching column variables from two awks!i have a program that prints out a formatted df command
by sorting and grepping and using awk to print it out.
it basically executes df -h | grep whatever | sort -n +5 and then
awks
to a nice printout like:
disk size capacity
------------------
foo 20gb 70%
moo 40gb 25%
bar 20gb 70%
i'd like to add a new column "accounts" to this print out.
the no. of accounts is derived from a command:
awk {'print $5'} myFile |sort |uniq -c |sort -nr
which prints rows like:
450 foo
300 moo
104 bar
i want to integrate this command into the formatted df program's
output.
i can't figure out how to appropriately append the "account count"
column from the myFile command, to the appropriate row of the
formatted df output,
e.g. where col 1 from the formatted df output matches col 2 of the
myFile command.
so it looks like this:
disk size capacity accounts
---------------------------
foo 20gb 70% 450
moo 40gb 25% 300
bar 20gb 70% 104
Thanks for any hints!
sg
Post Follow-up to this message
Re: matching column variables from two awks!
On 3/20/2008 11:36 PM, spacegoose wrote:
> i have a program that prints out a formatted df command
> by sorting and grepping and using awk to print it out.
>
> it basically executes df -h | grep whatever | sort -n +5 and then
> awks
> to a nice printout like:
>
> disk size capacity
> ------------------
> foo 20gb 70%
> moo 40gb 25%
> bar 20gb 70%
>
> i'd like to add a new column "accounts" to this print out.
>
> the no. of accounts is derived from a command:
> awk {'print $5'} myFile |sort |uniq -c |sort -nr
>
> which prints rows like:
>
> 450 foo
> 300 moo
> 104 bar
>
> i want to integrate this command into the formatted df program's
> output.
> i can't figure out how to appropriately append the "account count"
> column from the myFile command, to the appropriate row of the
> formatted df output,
> e.g. where col 1 from the formatted df output matches col 2 of the
> myFile command.
>
> so it looks like this:
>
>
> disk size capacity accounts
> ---------------------------
> foo 20gb 70% 450
> moo 40gb 25% 300
> bar 20gb 70% 104
>
>
> Thanks for any hints!
> sg
$ cat file1
450 foo
300 moo
104 bar
$ cat file2
disk size capacity
------------------
foo 20gb 70%
moo 40gb 25%
bar 20gb 70%
$ awk '
NR==FNR{acct[$2]=$1;next} FNR==1{sfx=" account"} FNR==2{sfx="--------"}
FNR>2{sfx=" "acct[$1]} {print $0 sfx}' file1 file2
disk size capacity account
--------------------------
foo 20gb 70% 450
moo 40gb 25% 300
bar 20gb 70% 104
but I expect there's a much simpler way to get your desired output using awk
on
your raw output rather than doing all that post-processing with other tools
first. If you provide your "df -h" output and "myFile" contents that got you
the
data above, we could probably help.
Ed.
Post Follow-up to this message Re: matching column variables from two awks!On Mar 21, 12:04 am, Ed Morton <mor...@lsupcaemnt.com> wrote:
> On 3/20/2008 11:36 PM, spacegoose wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> $ cat file1
> 450 foo
> 300 moo
> 104 bar
>
> $ cat file2
> disk size capacity
> ------------------
> foo 20gb 70%
> moo 40gb 25%
> bar 20gb 70%
>
> $ awk '
> NR==FNR{acct[$2]=$1;next} FNR==1{sfx=" account"} FNR==2{sfx="--------"}
> FNR>2{sfx=" "acct[$1]} {print $0 sfx}' file1 file2
> disk size capacity account
> --------------------------
> foo 20gb 70% 450
> moo 40gb 25% 300
> bar 20gb 70% 104
>
> but I expect there's a much simpler way to get your desired output using a
wk on
> your raw output rather than doing all that post-processing with other tool
s
> first. If you provide your "df -h" output and "myFile" contents that got y
ou the
> data above, we could probably help.
>
> Ed.
Thanks Ed,
I definitely want to work with the output rather than files (except
reading from myFile).
Fere's the df awk:
df -h | grep dsk | sort +5 | awk '
BEGIN {
printf("%-30s%12s%10s%10s\n\n", "File System", "bytes", "capacity",
"accounts")
printf("-------------------------------------------------------------------
\n")
T2 = 0
T3 = 0
T4 = 0
}
{
printf("%-30s%12s%10s%10s\n", $6, $2, $5, "# accts")
T2 += $2
T3 += $3
T4 += $4
}'
Post Follow-up to this message Re: matching column variables from two awks!
On 3/21/2008 9:15 AM, spacegoose wrote:
> On Mar 21, 12:04 am, Ed Morton <mor...@lsupcaemnt.com> wrote:
>
>
>
>
> Thanks Ed,
>
> I definitely want to work with the output rather than files (except
> reading from myFile).
> Fere's the df awk:
>
>
> df -h | grep dsk | sort +5 | awk '
>
> BEGIN {
> printf("%-30s%12s%10s%10s\n\n", "File System", "bytes", "capacity",
> "accounts")
> printf("------------------------------------------------------------------
-
> \n")
> T2 = 0
> T3 = 0
> T4 = 0
> }
>
> {
> printf("%-30s%12s%10s%10s\n", $6, $2, $5, "# accts")
> T2 += $2
> T3 += $3
> T4 += $4
> }'
>
You never use T2, T3, and T4.
I asked you to post the df -h output and the contents of MyFile so we're not
guessing but try this:
df -h | sort +5 | awk '
BEGIN {
printf "%-30s%12s%10s%10s\n\n", "File System", "bytes", "capacity", "account
s"
print "-------------------------------------------------------------------"
}
NR==FNR{ accts[$5]++; next }
{ printf "%-30s%12s%10s%10s\n", $6, $2, $5, accts[$6] }' MyFile -
If that doesn't do what you want, post what I suggested so we can see what
you're working with.
Ed.
Post Follow-up to this message Re: matching column variables from two awks!On Mar 21, 10:40 am, Ed Morton <mor...@lsupcaemnt.com> wrote:
> On 3/21/2008 9:15 AM, spacegoose wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> You never use T2, T3, and T4.
>
> I asked you to post the df -h output and the contents of MyFile so we're n
ot
> guessing but try this:
>
> df -h | sort +5 | awk '
> BEGIN {
> printf "%-30s%12s%10s%10s\n\n", "File System", "bytes", "capacity", "accou
nts"
> print "-------------------------------------------------------------------
"}
>
> NR==FNR{ accts[$5]++; next }
> { printf "%-30s%12s%10s%10s\n", $6, $2, $5, accts[$6] }' MyFile -
>
> If that doesn't do what you want, post what I suggested so we can see what
> you're working with.
>
> Ed.
Thanks for the help - the above did not work.
Here's the output of my df awk
File System bytes capacity accounts
-------------------------------------------------------------------
/ 19G 53%
/local/ds/xp0 200G 71%
/local/ds/xp1 200G 66%
/local/ds/xp10 200G 66%
/local/ds/xp11 200G 68%
/local/ds/xp12 200G 67%
here's a sample from myFile -
611 65114 2008/03/20 15:31 xp6 256000 user/xxx/INBOX
134 4805 2008/03/20 13:41 xp7 256000 user/yyy/INBOX
398 13403 2008/03/19 14:05 xp7 256000 user/zzz/INBOX
367 31508 2008/03/20 15:19 xp8 256000 user/aaa/INBOX
45 1759 2006/02/18 11:05 xp9 256000 user/bbb/INBOX
Of which:
I can derive the number of accts on each partition (xp#) from this:
awk {'print $5'} myFile |sort |uniq -c |sort -nr
A sample of its output looks like:
2145 xp30
2131 xp3
2129 xp33
2126 xp6
2123 xp32
I want to get the # in the first column to show up appropriately in
the df output (where the partitions, e.g.col2 here, matches col1 - I
now see there are /slashes/ to contend with...).
Thanks again,
sg
Post Follow-up to this message Re: matching column variables from two awks!
On 3/21/2008 11:33 AM, spacegoose wrote:
> On Mar 21, 10:40 am, Ed Morton <mor...@lsupcaemnt.com> wrote:
>
>
>
> Thanks for the help - the above did not work.
In what way? What did it produce vs what you wanted it to produce?
> Here's the output of my df awk
I'm asking for the output of df, not of "my df awk".
>
> File System bytes capacity accounts
>
> -------------------------------------------------------------------
> / 19G 53%
> /local/ds/xp0 200G 71%
> /local/ds/xp1 200G 66%
> /local/ds/xp10 200G 66%
> /local/ds/xp11 200G 68%
> /local/ds/xp12 200G 67%
>
>
> here's a sample from myFile -
>
> 611 65114 2008/03/20 15:31 xp6 256000 user/xxx/INBOX
> 134 4805 2008/03/20 13:41 xp7 256000 user/yyy/INBOX
> 398 13403 2008/03/19 14:05 xp7 256000 user/zzz/INBOX
> 367 31508 2008/03/20 15:19 xp8 256000 user/aaa/INBOX
> 45 1759 2006/02/18 11:05 xp9 256000 user/bbb/INBOX
I don't see the correlation between fields in "myFile" and the "my df awk"
output. Show a "myFile" that has some mapping to your df output and tell us
which field(s) should be used for that mapping.
>
> Of which:
> I can derive the number of accts on each partition (xp#) from this:
> awk {'print $5'} myFile |sort |uniq -c |sort -nr
Yes, but you can do it without so many pipes and different commands too.
> A sample of its output looks like:
>
> 2145 xp30
> 2131 xp3
> 2129 xp33
> 2126 xp6
> 2123 xp32
>
> I want to get the # in the first column to show up appropriately in
> the df output (where the partitions, e.g.col2 here, matches col1 - I
> now see there are /slashes/ to contend with...).
Why would slashes matter?
If you'd like help, just post the output of df (not the output of your scrip
t
that runs in a chain of pipes), the contents of myFile and your desired outp
ut.
Ed.
Post Follow-up to this message Re: matching column variables from two awks!On Mar 21, 3:59 pm, Ed Morton <mor...@lsupcaemnt.com> wrote: > On 3/21/2008 11:33 AM, spacegoose wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In what way? What did it produce vs what you wanted it to produce? It seemed to print out my entire myFile (over 100,000 lines). I wanted to only print the no. of accounts per partition (which that piped mishmash does). > > > I'm asking for the output of df, not of "my df awk". here a snip of df: /local/ds/xp30(/dev/md/ds/dsk/x30):133896070 blocks 22413779 files /local/ds/xp6 (/dev/md/ds/dsk/x6):121164266 blocks 22385181 files /local/ds/xp4 (/dev/md/ds/dsk/x4):129245812 blocks 22194814 files /local/ds/xp34(/dev/md/ds/dsk/x34):114122034 blocks 22075764 files /local/ds/xp22(/dev/md/ds/dsk/x22):109638490 blocks 22290938 files /local/ds/xp8 (/dev/md/ds/dsk/x8):132204190 blocks 22382085 files > > > > > > > > > > I don't see the correlation between fields in "myFile" and the "my df awk" > output. Show a "myFile" that has some mapping to your df output and tell u s > which field(s) should be used for that mapping. Initially, there wasn't since I'm snipping - now I have modified the above so there is a correlation. > > > > > Yes, but you can do it without so many pipes and different commands too. > > > > > Why would slashes matter? > > If you'd like help, just post the output of df (not the output of your scr ipt > that runs in a chain of pipes), the contents of myFile and your desired output.[/c olor] myFile sample is here: 611 65114 2008/03/20 15:31 xp0 256000 user/xxx/INBOX 134 4805 2008/03/20 13:41 xp1 256000 user/yyy/INBOX 398 13403 2008/03/19 14:05 xp10 256000 user/zzz/INBOX 367 31508 2008/03/20 15:19 xp11 256000 user/aaa/INBOX 45 1759 2006/02/18 11:05 xp12 256000 user/bbb/INBOX desired out: File System bytes capacity accounts ------------------------------------------------------------------- / 19G 53% 2150 /local/ds/xp0 200G 71% 2250 /local/ds/xp1 200G 66% 2167 /local/ds/xp10 200G 66% 3400 /local/ds/xp11 200G 68% 4444 /local/ds/xp12 200G 67% 5000 > > Ed. Thanks! sg
Post Follow-up to this message Re: matching column variables from two awks!On Mar 21, 4:51 pm, spacegoose <spacego...@gmail.com> wrote: > On Mar 21, 3:59 pm, Ed Morton <mor...@lsupcaemnt.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It seemed to print out my entire myFile (over 100,000 lines). I wanted > to only print the no. of accounts per partition (which that piped > mishmash does). > > > > > > here a snip of df: > /local/ds/xp30(/dev/md/ds/dsk/x30):133896070 blocks 22413779 files > /local/ds/xp6 (/dev/md/ds/dsk/x6):121164266 blocks 22385181 files > /local/ds/xp4 (/dev/md/ds/dsk/x4):129245812 blocks 22194814 files > /local/ds/xp34(/dev/md/ds/dsk/x34):114122034 blocks 22075764 files > /local/ds/xp22(/dev/md/ds/dsk/x22):109638490 blocks 22290938 files > /local/ds/xp8 (/dev/md/ds/dsk/x8):132204190 blocks 22382085 files > > > > > > > > > > > Initially, there wasn't since I'm snipping - now I have modified the > above so there is a correlation. > > > > > > > > > > > > > myFile sample is here: > 611 65114 2008/03/20 15:31 xp0 256000 user/xxx/INBOX > 134 4805 2008/03/20 13:41 xp1 256000 user/yyy/INBOX > 398 13403 2008/03/19 14:05 xp10 256000 user/zzz/INBOX > 367 31508 2008/03/20 15:19 xp11 256000 user/aaa/INBOX > 45 1759 2006/02/18 11:05 xp12 256000 user/bbb/INBOX > > desired out: > File System bytes capacity accounts > ------------------------------------------------------------------- > / 19G 53% 2150 > /local/ds/xp0 200G 71% 2250 > /local/ds/xp1 200G 66% 2167 > /local/ds/xp10 200G 66% 3400 > /local/ds/xp11 200G 68% 4444 > /local/ds/xp12 200G 67% 5000 > > > > > Thanks! > sg here is some more df output: xxx@x3$ df -h | grep dsk | sort +5 Filesystem size used avail capacity Mounted on /dev/md/ds/dsk/d0 200G 139G 59G 71% /local/ds/xp0 /dev/md/ds/dsk/d1 200G 131G 68G 66% /local/ds/xp1 /dev/md/ds/dsk/d10 200G 129G 69G 66% /local/ds/xp10 /dev/md/ds/dsk/d11 200G 133G 65G 68% /local/ds/xp11 /dev/md/ds/dsk/d12 200G 131G 67G 67% /local/ds/xp12 xxx@x3$ df -h Filesystem size used avail capacity Mounted on /dev/md/ds/dsk/x18 200G 133G 65G 68% /local/ds/xp18 /dev/md/ds/dsk/x14 200G 139G 59G 71% /local/ds/xp14 /dev/md/ds/dsk/x16 200G 138G 60G 70% /local/ds/xp16 /dev/md/ds/dsk/x2 200G 130G 68G 66% /local/ds/xp2 /dev/md/ds/dsk/x26 200G 129G 69G 66% /local/ds/xp26 /dev/md/ds/dsk/x12
Post Follow-up to this message Re: matching column variables from two awks!On 3/21/2008 3:51 PM, spacegoose wrote:
<snip>
> here a snip of df:
> /local/ds/xp30(/dev/md/ds/dsk/x30):133896070 blocks 22413779 files
> /local/ds/xp6 (/dev/md/ds/dsk/x6):121164266 blocks 22385181 files
> /local/ds/xp4 (/dev/md/ds/dsk/x4):129245812 blocks 22194814 files
> /local/ds/xp34(/dev/md/ds/dsk/x34):114122034 blocks 22075764 files
> /local/ds/xp22(/dev/md/ds/dsk/x22):109638490 blocks 22290938 files
> /local/ds/xp8 (/dev/md/ds/dsk/x8):132204190 blocks 22382085 files
Hmm, that doesn't look like what I get with "df -h". Oh well. So, sometimes
there's a space before the first "(":
/local/ds/xp6 (/dev/md/...
and sometimes there isn't:
/local/ds/xp30(/dev/md
Right?
<snip>
> myFile sample is here:
> 611 65114 2008/03/20 15:31 xp0 256000 user/xxx/INBOX
> 134 4805 2008/03/20 13:41 xp1 256000 user/yyy/INBOX
> 398 13403 2008/03/19 14:05 xp10 256000 user/zzz/INBOX
> 367 31508 2008/03/20 15:19 xp11 256000 user/aaa/INBOX
> 45 1759 2006/02/18 11:05 xp12 256000 user/bbb/INBOX
So, the 5th field (xp0, xp1, etc.) is the key to match to the end of the fir
st
field in the df output (/local/ds/xp6, etc.). i.e. I just need to prepend
"/local/ds/" to the 5th field of myFile to get the first field of the df out
put.
Right?
> desired out:
> File System bytes capacity accounts
> -------------------------------------------------------------------
> / 19G 53% 2150
> /local/ds/xp0 200G 71% 2250
> /local/ds/xp1 200G 66% 2167
> /local/ds/xp10 200G 66% 3400
> /local/ds/xp11 200G 68% 4444
> /local/ds/xp12 200G 67% 5000
How are you getting those bytes and capacity numbers from the df output you
posted?
PLEASE post the input that could actually produce the desired output if you'
d
like help.
Ed.
Post Follow-up to this message Re: matching column variables from two awks!On Mar 21, 5:16 pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
> On 3/21/2008 3:51 PM, spacegoose wrote:
> <snip>
>
>
> Hmm, that doesn't look like what I get with "df -h". Oh well. So, sometime
s
> there's a space before the first "(":
>
> /local/ds/xp6 (/dev/md/...
>
> and sometimes there isn't:
>
> /local/ds/xp30(/dev/md
yes.
>
> Right?
>
> <snip>
>
>
> So, the 5th field (xp0, xp1, etc.) is the key to match to the end of the f
irst
> field in the df output (/local/ds/xp6, etc.). i.e. I just need to prepend
> "/local/ds/" to the 5th field of myFile to get the first field of the df o
utput.
> Right?
yes.
>
>
> How are you getting those bytes and capacity numbers from the df output you posted
?
i'm getting those fields with: df -h | grep dsk | sort +5
>
> PLEASE post the input that could actually produce the desired output if yo
u'd
> like help.
>
> Ed.
thanks for your help - i will try to make this clearer soon.
Post Follow-up to this message
Sponsored Links
Powered by vBulletin
Copyright 2000-2006 Jelsoft Enterprises Limited.
