Home > Archive > AWK > March 2008 > matching column variables from two awks!
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
matching column variables from two awks!
|
|
| spacegoose 2008-03-21, 3:59 am |
| i have a program that prints out a formatted df command
by sorting and grepping and using awk to print it out.
it basically executes df -h | grep whatever | sort -n +5 and then
awks
to a nice printout like:
disk size capacity
------------------
foo 20gb 70%
moo 40gb 25%
bar 20gb 70%
i'd like to add a new column "accounts" to this print out.
the no. of accounts is derived from a command:
awk {'print $5'} myFile |sort |uniq -c |sort -nr
which prints rows like:
450 foo
300 moo
104 bar
i want to integrate this command into the formatted df program's
output.
i can't figure out how to appropriately append the "account count"
column from the myFile command, to the appropriate row of the
formatted df output,
e.g. where col 1 from the formatted df output matches col 2 of the
myFile command.
so it looks like this:
disk size capacity accounts
---------------------------
foo 20gb 70% 450
moo 40gb 25% 300
bar 20gb 70% 104
Thanks for any hints!
sg
| |
| Ed Morton 2008-03-21, 3:59 am |
|
On 3/20/2008 11:36 PM, spacegoose wrote:
> i have a program that prints out a formatted df command
> by sorting and grepping and using awk to print it out.
>
> it basically executes df -h | grep whatever | sort -n +5 and then
> awks
> to a nice printout like:
>
> disk size capacity
> ------------------
> foo 20gb 70%
> moo 40gb 25%
> bar 20gb 70%
>
> i'd like to add a new column "accounts" to this print out.
>
> the no. of accounts is derived from a command:
> awk {'print $5'} myFile |sort |uniq -c |sort -nr
>
> which prints rows like:
>
> 450 foo
> 300 moo
> 104 bar
>
> i want to integrate this command into the formatted df program's
> output.
> i can't figure out how to appropriately append the "account count"
> column from the myFile command, to the appropriate row of the
> formatted df output,
> e.g. where col 1 from the formatted df output matches col 2 of the
> myFile command.
>
> so it looks like this:
>
>
> disk size capacity accounts
> ---------------------------
> foo 20gb 70% 450
> moo 40gb 25% 300
> bar 20gb 70% 104
>
>
> Thanks for any hints!
> sg
$ cat file1
450 foo
300 moo
104 bar
$ cat file2
disk size capacity
------------------
foo 20gb 70%
moo 40gb 25%
bar 20gb 70%
$ awk '
NR==FNR{acct[$2]=$1;next} FNR==1{sfx=" account"} FNR==2{sfx="--------"}
FNR>2{sfx=" "acct[$1]} {print $0 sfx}' file1 file2
disk size capacity account
--------------------------
foo 20gb 70% 450
moo 40gb 25% 300
bar 20gb 70% 104
but I expect there's a much simpler way to get your desired output using awk on
your raw output rather than doing all that post-processing with other tools
first. If you provide your "df -h" output and "myFile" contents that got you the
data above, we could probably help.
Ed.
| |
| spacegoose 2008-03-21, 6:59 pm |
| On Mar 21, 12:04 am, Ed Morton <mor...@lsupcaemnt.com> wrote:
> On 3/20/2008 11:36 PM, spacegoose wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> $ cat file1
> 450 foo
> 300 moo
> 104 bar
>
> $ cat file2
> disk size capacity
> ------------------
> foo 20gb 70%
> moo 40gb 25%
> bar 20gb 70%
>
> $ awk '
> NR==FNR{acct[$2]=$1;next} FNR==1{sfx=" account"} FNR==2{sfx="--------"}
> FNR>2{sfx=" "acct[$1]} {print $0 sfx}' file1 file2
> disk size capacity account
> --------------------------
> foo 20gb 70% 450
> moo 40gb 25% 300
> bar 20gb 70% 104
>
> but I expect there's a much simpler way to get your desired output using awk on
> your raw output rather than doing all that post-processing with other tools
> first. If you provide your "df -h" output and "myFile" contents that got you the
> data above, we could probably help.
>
> Ed.
Thanks Ed,
I definitely want to work with the output rather than files (except
reading from myFile).
Fere's the df awk:
df -h | grep dsk | sort +5 | awk '
BEGIN {
printf("%-30s%12s%10s%10s\n\n", "File System", "bytes", "capacity",
"accounts")
printf("-------------------------------------------------------------------
\n")
T2 = 0
T3 = 0
T4 = 0
}
{
printf("%-30s%12s%10s%10s\n", $6, $2, $5, "# accts")
T2 += $2
T3 += $3
T4 += $4
}'
| |
| Ed Morton 2008-03-21, 6:59 pm |
|
On 3/21/2008 9:15 AM, spacegoose wrote:
> On Mar 21, 12:04 am, Ed Morton <mor...@lsupcaemnt.com> wrote:
>
>
>
>
> Thanks Ed,
>
> I definitely want to work with the output rather than files (except
> reading from myFile).
> Fere's the df awk:
>
>
> df -h | grep dsk | sort +5 | awk '
>
> BEGIN {
> printf("%-30s%12s%10s%10s\n\n", "File System", "bytes", "capacity",
> "accounts")
> printf("-------------------------------------------------------------------
> \n")
> T2 = 0
> T3 = 0
> T4 = 0
> }
>
> {
> printf("%-30s%12s%10s%10s\n", $6, $2, $5, "# accts")
> T2 += $2
> T3 += $3
> T4 += $4
> }'
>
You never use T2, T3, and T4.
I asked you to post the df -h output and the contents of MyFile so we're not
guessing but try this:
df -h | sort +5 | awk '
BEGIN {
printf "%-30s%12s%10s%10s\n\n", "File System", "bytes", "capacity", "accounts"
print "-------------------------------------------------------------------"
}
NR==FNR{ accts[$5]++; next }
{ printf "%-30s%12s%10s%10s\n", $6, $2, $5, accts[$6] }' MyFile -
If that doesn't do what you want, post what I suggested so we can see what
you're working with.
Ed.
| |
| spacegoose 2008-03-21, 6:59 pm |
| On Mar 21, 10:40 am, Ed Morton <mor...@lsupcaemnt.com> wrote:
> On 3/21/2008 9:15 AM, spacegoose wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> You never use T2, T3, and T4.
>
> I asked you to post the df -h output and the contents of MyFile so we're not
> guessing but try this:
>
> df -h | sort +5 | awk '
> BEGIN {
> printf "%-30s%12s%10s%10s\n\n", "File System", "bytes", "capacity", "accounts"
> print "-------------------------------------------------------------------"}
>
> NR==FNR{ accts[$5]++; next }
> { printf "%-30s%12s%10s%10s\n", $6, $2, $5, accts[$6] }' MyFile -
>
> If that doesn't do what you want, post what I suggested so we can see what
> you're working with.
>
> Ed.
Thanks for the help - the above did not work.
Here's the output of my df awk
File System bytes capacity accounts
-------------------------------------------------------------------
/ 19G 53%
/local/ds/xp0 200G 71%
/local/ds/xp1 200G 66%
/local/ds/xp10 200G 66%
/local/ds/xp11 200G 68%
/local/ds/xp12 200G 67%
here's a sample from myFile -
611 65114 2008/03/20 15:31 xp6 256000 user/xxx/INBOX
134 4805 2008/03/20 13:41 xp7 256000 user/yyy/INBOX
398 13403 2008/03/19 14:05 xp7 256000 user/zzz/INBOX
367 31508 2008/03/20 15:19 xp8 256000 user/aaa/INBOX
45 1759 2006/02/18 11:05 xp9 256000 user/bbb/INBOX
Of which:
I can derive the number of accts on each partition (xp#) from this:
awk {'print $5'} myFile |sort |uniq -c |sort -nr
A sample of its output looks like:
2145 xp30
2131 xp3
2129 xp33
2126 xp6
2123 xp32
I want to get the # in the first column to show up appropriately in
the df output (where the partitions, e.g.col2 here, matches col1 - I
now see there are /slashes/ to contend with...).
Thanks again,
sg
| |
| Ed Morton 2008-03-21, 6:59 pm |
|
On 3/21/2008 11:33 AM, spacegoose wrote:
> On Mar 21, 10:40 am, Ed Morton <mor...@lsupcaemnt.com> wrote:
>
>
>
> Thanks for the help - the above did not work.
In what way? What did it produce vs what you wanted it to produce?
> Here's the output of my df awk
I'm asking for the output of df, not of "my df awk".
>
> File System bytes capacity accounts
>
> -------------------------------------------------------------------
> / 19G 53%
> /local/ds/xp0 200G 71%
> /local/ds/xp1 200G 66%
> /local/ds/xp10 200G 66%
> /local/ds/xp11 200G 68%
> /local/ds/xp12 200G 67%
>
>
> here's a sample from myFile -
>
> 611 65114 2008/03/20 15:31 xp6 256000 user/xxx/INBOX
> 134 4805 2008/03/20 13:41 xp7 256000 user/yyy/INBOX
> 398 13403 2008/03/19 14:05 xp7 256000 user/zzz/INBOX
> 367 31508 2008/03/20 15:19 xp8 256000 user/aaa/INBOX
> 45 1759 2006/02/18 11:05 xp9 256000 user/bbb/INBOX
I don't see the correlation between fields in "myFile" and the "my df awk"
output. Show a "myFile" that has some mapping to your df output and tell us
which field(s) should be used for that mapping.
>
> Of which:
> I can derive the number of accts on each partition (xp#) from this:
> awk {'print $5'} myFile |sort |uniq -c |sort -nr
Yes, but you can do it without so many pipes and different commands too.
> A sample of its output looks like:
>
> 2145 xp30
> 2131 xp3
> 2129 xp33
> 2126 xp6
> 2123 xp32
>
> I want to get the # in the first column to show up appropriately in
> the df output (where the partitions, e.g.col2 here, matches col1 - I
> now see there are /slashes/ to contend with...).
Why would slashes matter?
If you'd like help, just post the output of df (not the output of your script
that runs in a chain of pipes), the contents of myFile and your desired output.
Ed.
| |
| spacegoose 2008-03-21, 6:59 pm |
| On Mar 21, 3:59 pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
> On 3/21/2008 11:33 AM, spacegoose wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> In what way? What did it produce vs what you wanted it to produce?
It seemed to print out my entire myFile (over 100,000 lines). I wanted
to only print the no. of accounts per partition (which that piped
mishmash does).
>
>
> I'm asking for the output of df, not of "my df awk".
here a snip of df:
/local/ds/xp30(/dev/md/ds/dsk/x30):133896070 blocks 22413779 files
/local/ds/xp6 (/dev/md/ds/dsk/x6):121164266 blocks 22385181 files
/local/ds/xp4 (/dev/md/ds/dsk/x4):129245812 blocks 22194814 files
/local/ds/xp34(/dev/md/ds/dsk/x34):114122034 blocks 22075764 files
/local/ds/xp22(/dev/md/ds/dsk/x22):109638490 blocks 22290938 files
/local/ds/xp8 (/dev/md/ds/dsk/x8):132204190 blocks 22382085 files
>
>
>
>
>
>
>
>
>
> I don't see the correlation between fields in "myFile" and the "my df awk"
> output. Show a "myFile" that has some mapping to your df output and tell us
> which field(s) should be used for that mapping.
Initially, there wasn't since I'm snipping - now I have modified the
above so there is a correlation.
>
>
>
>
> Yes, but you can do it without so many pipes and different commands too.
>
>
>
>
> Why would slashes matter?
>
> If you'd like help, just post the output of df (not the output of your script
> that runs in a chain of pipes), the contents of myFile and your desired output.
myFile sample is here:
611 65114 2008/03/20 15:31 xp0 256000 user/xxx/INBOX
134 4805 2008/03/20 13:41 xp1 256000 user/yyy/INBOX
398 13403 2008/03/19 14:05 xp10 256000 user/zzz/INBOX
367 31508 2008/03/20 15:19 xp11 256000 user/aaa/INBOX
45 1759 2006/02/18 11:05 xp12 256000 user/bbb/INBOX
desired out:
File System bytes capacity accounts
-------------------------------------------------------------------
/ 19G 53% 2150
/local/ds/xp0 200G 71% 2250
/local/ds/xp1 200G 66% 2167
/local/ds/xp10 200G 66% 3400
/local/ds/xp11 200G 68% 4444
/local/ds/xp12 200G 67% 5000
>
> Ed.
Thanks!
sg
| |
| spacegoose 2008-03-21, 6:59 pm |
| On Mar 21, 4:51 pm, spacegoose <spacego...@gmail.com> wrote:
> On Mar 21, 3:59 pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> It seemed to print out my entire myFile (over 100,000 lines). I wanted
> to only print the no. of accounts per partition (which that piped
> mishmash does).
>
>
>
>
>
> here a snip of df:
> /local/ds/xp30(/dev/md/ds/dsk/x30):133896070 blocks 22413779 files
> /local/ds/xp6 (/dev/md/ds/dsk/x6):121164266 blocks 22385181 files
> /local/ds/xp4 (/dev/md/ds/dsk/x4):129245812 blocks 22194814 files
> /local/ds/xp34(/dev/md/ds/dsk/x34):114122034 blocks 22075764 files
> /local/ds/xp22(/dev/md/ds/dsk/x22):109638490 blocks 22290938 files
> /local/ds/xp8 (/dev/md/ds/dsk/x8):132204190 blocks 22382085 files
>
>
>
>
>
>
>
>
>
>
> Initially, there wasn't since I'm snipping - now I have modified the
> above so there is a correlation.
>
>
>
>
>
>
>
>
>
>
>
>
> myFile sample is here:
> 611 65114 2008/03/20 15:31 xp0 256000 user/xxx/INBOX
> 134 4805 2008/03/20 13:41 xp1 256000 user/yyy/INBOX
> 398 13403 2008/03/19 14:05 xp10 256000 user/zzz/INBOX
> 367 31508 2008/03/20 15:19 xp11 256000 user/aaa/INBOX
> 45 1759 2006/02/18 11:05 xp12 256000 user/bbb/INBOX
>
> desired out:
> File System bytes capacity accounts
> -------------------------------------------------------------------
> / 19G 53% 2150
> /local/ds/xp0 200G 71% 2250
> /local/ds/xp1 200G 66% 2167
> /local/ds/xp10 200G 66% 3400
> /local/ds/xp11 200G 68% 4444
> /local/ds/xp12 200G 67% 5000
>
>
>
>
> Thanks!
> sg
here is some more df output:
xxx@x3$ df -h | grep dsk | sort +5
Filesystem size used avail capacity Mounted on
/dev/md/ds/dsk/d0 200G 139G 59G 71% /local/ds/xp0
/dev/md/ds/dsk/d1 200G 131G 68G 66% /local/ds/xp1
/dev/md/ds/dsk/d10 200G 129G 69G 66% /local/ds/xp10
/dev/md/ds/dsk/d11 200G 133G 65G 68% /local/ds/xp11
/dev/md/ds/dsk/d12 200G 131G 67G 67% /local/ds/xp12
xxx@x3$ df -h
Filesystem size used avail capacity Mounted on
/dev/md/ds/dsk/x18
200G 133G 65G 68% /local/ds/xp18
/dev/md/ds/dsk/x14
200G 139G 59G 71% /local/ds/xp14
/dev/md/ds/dsk/x16
200G 138G 60G 70% /local/ds/xp16
/dev/md/ds/dsk/x2
200G 130G 68G 66% /local/ds/xp2
/dev/md/ds/dsk/x26
200G 129G 69G 66% /local/ds/xp26
/dev/md/ds/dsk/x12
| |
| Ed Morton 2008-03-21, 6:59 pm |
| On 3/21/2008 3:51 PM, spacegoose wrote:
<snip>
> here a snip of df:
> /local/ds/xp30(/dev/md/ds/dsk/x30):133896070 blocks 22413779 files
> /local/ds/xp6 (/dev/md/ds/dsk/x6):121164266 blocks 22385181 files
> /local/ds/xp4 (/dev/md/ds/dsk/x4):129245812 blocks 22194814 files
> /local/ds/xp34(/dev/md/ds/dsk/x34):114122034 blocks 22075764 files
> /local/ds/xp22(/dev/md/ds/dsk/x22):109638490 blocks 22290938 files
> /local/ds/xp8 (/dev/md/ds/dsk/x8):132204190 blocks 22382085 files
Hmm, that doesn't look like what I get with "df -h". Oh well. So, sometimes
there's a space before the first "(":
/local/ds/xp6 (/dev/md/...
and sometimes there isn't:
/local/ds/xp30(/dev/md
Right?
<snip>
> myFile sample is here:
> 611 65114 2008/03/20 15:31 xp0 256000 user/xxx/INBOX
> 134 4805 2008/03/20 13:41 xp1 256000 user/yyy/INBOX
> 398 13403 2008/03/19 14:05 xp10 256000 user/zzz/INBOX
> 367 31508 2008/03/20 15:19 xp11 256000 user/aaa/INBOX
> 45 1759 2006/02/18 11:05 xp12 256000 user/bbb/INBOX
So, the 5th field (xp0, xp1, etc.) is the key to match to the end of the first
field in the df output (/local/ds/xp6, etc.). i.e. I just need to prepend
"/local/ds/" to the 5th field of myFile to get the first field of the df output.
Right?
> desired out:
> File System bytes capacity accounts
> -------------------------------------------------------------------
> / 19G 53% 2150
> /local/ds/xp0 200G 71% 2250
> /local/ds/xp1 200G 66% 2167
> /local/ds/xp10 200G 66% 3400
> /local/ds/xp11 200G 68% 4444
> /local/ds/xp12 200G 67% 5000
How are you getting those bytes and capacity numbers from the df output you posted?
PLEASE post the input that could actually produce the desired output if you'd
like help.
Ed.
| |
| spacegoose 2008-03-21, 6:59 pm |
| On Mar 21, 5:16 pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
> On 3/21/2008 3:51 PM, spacegoose wrote:
> <snip>
>
>
> Hmm, that doesn't look like what I get with "df -h". Oh well. So, sometimes
> there's a space before the first "(":
>
> /local/ds/xp6 (/dev/md/...
>
> and sometimes there isn't:
>
> /local/ds/xp30(/dev/md
yes.
>
> Right?
>
> <snip>
>
>
> So, the 5th field (xp0, xp1, etc.) is the key to match to the end of the first
> field in the df output (/local/ds/xp6, etc.). i.e. I just need to prepend
> "/local/ds/" to the 5th field of myFile to get the first field of the df output.
> Right?
yes.
>
>
> How are you getting those bytes and capacity numbers from the df output you posted?
i'm getting those fields with: df -h | grep dsk | sort +5
>
> PLEASE post the input that could actually produce the desired output if you'd
> like help.
>
> Ed.
thanks for your help - i will try to make this clearer soon.
| |
| spacegoose 2008-03-21, 6:59 pm |
| xxx@x3$ df -h | grep dsk | sort +5
/dev/md/ds/dsk/x20 200G 129G 69G 66% /local/ds/xp20
/dev/md/ds/dsk/x21 200G 135G 63G 69% /local/ds/xp21
/dev/md/ds/dsk/x22 200G 148G 50G 75% /local/ds/xp22
/dev/md/ds/dsk/x23 200G 128G 70G 65% /local/ds/xp23
/dev/md/ds/dsk/x24 200G 126G 72G 64% /local/ds/xp24
/dev/md/ds/dsk/x25 200G 136G 62G 69% /local/ds/xp25
/dev/md/ds/dsk/x26 200G 129G 69G 66% /local/ds/xp26
/dev/md/ds/dsk/x27 200G 130G 68G 66% /local/ds/xp27
xxx@x3$ df
/local/ds/xp26(/dev/md/ds/dsk/x26):148445868 blocks 22517420 files
/local/ds/xp12(/dev/md/ds/dsk/x12):144276990 blocks 22608345 files
/local/ds/xp30(/dev/md/ds/dsk/x30):133775594 blocks 22412624 files
/local/ds/xp6 (/dev/md/ds/dsk/x6):121057960 blocks 22383983 files
/local/ds/xp4 (/dev/md/ds/dsk/x4):129103034 blocks 22193600 files
my program foo:
df -h | grep dsk | sort +5 | awk '
BEGIN {
printf("%-30s%12s%12s%12s%10s\n\n", "File System", "bytes", "used",
"available", "capacity")
printf("------------------------------------------------------------------------------
\n")
}
{
printf("%-30s%12s%12s%12s%10s\n", $6, $2, $3, $4, $5)
}
END {
}'
xxx@x3$ ./foo
File System bytes used available
capacity
------------------------------------------------------------------------------
/global/ds/xp0 200G 139G 59G
71%
/global/ds/xp1 200G 131G 67G
66%
/global/ds/xp10 200G 129G 69G
66%
/global/ds/xp11 200G 133G 65G
68%
/global/ds/xp12 200G 131G 67G
67%
/global/ds/xp13 200G 139G 59G
71%
/global/ds/xp14 200G 139G 59G
71%
/global/ds/xp15 200G 132G 66G
67%
/global/ds/xp16 200G 138G 60G
70%
/global/ds/xp17 200G 128G 70G
65%
/global/ds/xp18 200G 133G 65G
68%
sample of myFile:
11 621 2008/03/21 12:44 xp4 256000 user/foo/INBOX
806 41722 2008/03/21 13:26 xp47 256000 user/bar/INBOX
2391 115428 2008/03/21 13:26 xp5 256000 user/moo/INBOX
2452 122771 2008/03/21 12:38 xp6 256000 user/123/INBOX
638 65484 2008/03/21 12:56 xp6 256000 user/345/INBOX
139 4892 2008/03/21 12:44 xp7 256000 user/789/INBOX
398 13403 2008/03/19 14:05 xp7 256000 user/1234/INBOX
392 32448 2008/03/21 14:14 xp8 256000 user/5678/INBOX
of which this:
xxx@x3$ awk {'print $5'} myFile | sort |uniq -c | sort -n
shows:
1407 xp45
1496 xp40
1503 xp51
1524 xp49
1537 xp50
1539 xp42
where col 1 is the no. of accts. per "xp" partition.
my desired output will show col 1 (as above) on the appropriate row
(e.g. where partition matches, e.g. where col 2 above matches the end
of col 1 from my prog's out)
xxx@x3$ ./foo
File System bytes used available
capacity accts
------------------------------------------------------------------------------
/global/ds/xp45 200G 139G 59G
71% 1407
/global/ds/xp40 200G 131G 67G
66% 1496
/global/ds/xp51 200G 129G 69G
66% 1503
/global/ds/xp49 200G 133G 65G
68% 1524
Thanks again!
sg
| |
| Ed Morton 2008-03-21, 6:59 pm |
| OK, let's try it this way. Look:
$ cat df_out
/dev/md/ds/dsk/x20 200G 129G 69G 66% /local/ds/xp20
/dev/md/ds/dsk/x21 200G 135G 63G 69% /local/ds/xp21
/dev/md/ds/dsk/x22 200G 148G 50G 75% /local/ds/xp22
/dev/md/ds/dsk/x23 200G 128G 70G 65% /local/ds/xp23
/dev/md/ds/dsk/x24 200G 126G 72G 64% /local/ds/xp24
/dev/md/ds/dsk/x25 200G 136G 62G 69% /local/ds/xp25
/dev/md/ds/dsk/x26 200G 129G 69G 66% /local/ds/xp26
/dev/md/ds/dsk/x27 200G 130G 68G 66% /local/ds/xp27
$
$ cat myFile
11 621 2008/03/21 12:44 xp20 256000 user/foo/INBOX
806 41722 2008/03/21 13:26 xp21 256000 user/bar/INBOX
2391 115428 2008/03/21 13:26 xp22 256000 user/moo/INBOX
2452 122771 2008/03/21 12:38 xp23 256000 user/123/INBOX
638 65484 2008/03/21 12:56 xp23 256000 user/345/INBOX
139 4892 2008/03/21 12:44 xp24 256000 user/789/INBOX
398 13403 2008/03/19 14:05 xp24 256000 user/1234/INBOX
392 32448 2008/03/21 14:14 xp25 256000 user/5678/INBOX
$
$ cat tst.awk
BEGIN {
printf "%-15s%12s%10s%10s%10s%10s\n", "File System", "bytes", "used", "avail",
"capacity", "accounts"
print "-------------------------------------------------------------------"
}
NR==FNR{ accts["/local/ds/"$5]++; next }
/dsk/ { printf "%-15s%12s%10s%10s%10s%10s\n", $6, $2, $3, $4, $5, accts[$6]+0 }
$
$ cat df_out | awk -f tst.awk myFile -
File System bytes used avail capacity accounts
-------------------------------------------------------------------
/local/ds/xp20 200G 129G 69G 66% 1
/local/ds/xp21 200G 135G 63G 69% 1
/local/ds/xp22 200G 148G 50G 75% 1
/local/ds/xp23 200G 128G 70G 65% 2
/local/ds/xp24 200G 126G 72G 64% 2
/local/ds/xp25 200G 136G 62G 69% 1
/local/ds/xp26 200G 129G 69G 66% 0
/local/ds/xp27 200G 130G 68G 66% 0
Is that what you wanted?
If so, just replace "cat df_out" with "df -h | sort +5" (no need for grep):
df -h | sort +5 | awk -f tst.awk myFile -
If not, what are you looking for?
Ed.
| |
| spacegoose 2008-03-22, 3:58 am |
| On Mar 21, 6:32 pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
> OK, let's try it this way. Look:
>
> $ cat df_out
> /dev/md/ds/dsk/x20 200G 129G 69G 66% /local/ds/xp20
> /dev/md/ds/dsk/x21 200G 135G 63G 69% /local/ds/xp21
> /dev/md/ds/dsk/x22 200G 148G 50G 75% /local/ds/xp22
> /dev/md/ds/dsk/x23 200G 128G 70G 65% /local/ds/xp23
> /dev/md/ds/dsk/x24 200G 126G 72G 64% /local/ds/xp24
> /dev/md/ds/dsk/x25 200G 136G 62G 69% /local/ds/xp25
> /dev/md/ds/dsk/x26 200G 129G 69G 66% /local/ds/xp26
> /dev/md/ds/dsk/x27 200G 130G 68G 66% /local/ds/xp27
> $
> $ cat myFile
> 11 621 2008/03/21 12:44 xp20 256000 user/foo/INBOX
> 806 41722 2008/03/21 13:26 xp21 256000 user/bar/INBOX
> 2391 115428 2008/03/21 13:26 xp22 256000 user/moo/INBOX
> 2452 122771 2008/03/21 12:38 xp23 256000 user/123/INBOX
> 638 65484 2008/03/21 12:56 xp23 256000 user/345/INBOX
> 139 4892 2008/03/21 12:44 xp24 256000 user/789/INBOX
> 398 13403 2008/03/19 14:05 xp24 256000 user/1234/INBOX
> 392 32448 2008/03/21 14:14 xp25 256000 user/5678/INBOX
> $
> $ cat tst.awk
> BEGIN {
> printf "%-15s%12s%10s%10s%10s%10s\n", "File System", "bytes", "used", "avail",
> "capacity", "accounts"
> print "-------------------------------------------------------------------"}
>
> NR==FNR{ accts["/local/ds/"$5]++; next }
> /dsk/ { printf "%-15s%12s%10s%10s%10s%10s\n", $6, $2, $3, $4, $5, accts[$6]+0 }
> $
> $ cat df_out | awk -f tst.awk myFile -
> File System bytes used avail capacity accounts
> -------------------------------------------------------------------
> /local/ds/xp20 200G 129G 69G 66% 1
> /local/ds/xp21 200G 135G 63G 69% 1
> /local/ds/xp22 200G 148G 50G 75% 1
> /local/ds/xp23 200G 128G 70G 65% 2
> /local/ds/xp24 200G 126G 72G 64% 2
> /local/ds/xp25 200G 136G 62G 69% 1
> /local/ds/xp26 200G 129G 69G 66% 0
> /local/ds/xp27 200G 130G 68G 66% 0
>
> Is that what you wanted?
yes - it's what i want - but making those files on my machine - and
running:
cat df_out | awk -f tst.awk myFile -
produces this:
File System bytes used avail capacity accounts
-------------------------------------------------------------------
/local/ds/xp20 200G 129G 69G 66% 0
/local/ds/xp21 200G 135G 63G 69% 0
/local/ds/xp22 200G 148G 50G 75% 0
/local/ds/xp23 200G 128G 70G 65% 0
/local/ds/xp24 200G 126G 72G 64% 0
/local/ds/xp25 200G 136G 62G 69% 0
/local/ds/xp26 200G 129G 69G 66% 0
/local/ds/xp27 200G 130G 68G 66% 0
>
> If so, just replace "cat df_out" with "df -h | sort +5" (no need for grep):
>
> df -h | sort +5 | awk -f tst.awk myFile -
>
> If not, what are you looking for?
>
> Ed.
| |
| Ed Morton 2008-03-22, 7:58 am |
|
On 3/22/2008 12:00 AM, spacegoose wrote:
> On Mar 21, 6:32 pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
>
>
>
> yes - it's what i want - but making those files on my machine - and
> running:
>
> cat df_out | awk -f tst.awk myFile -
>
> produces this:
>
>
>
> File System bytes used avail capacity accounts
> -------------------------------------------------------------------
> /local/ds/xp20 200G 129G 69G 66% 0
> /local/ds/xp21 200G 135G 63G 69% 0
> /local/ds/xp22 200G 148G 50G 75% 0
> /local/ds/xp23 200G 128G 70G 65% 0
> /local/ds/xp24 200G 126G 72G 64% 0
> /local/ds/xp25 200G 136G 62G 69% 0
> /local/ds/xp26 200G 129G 69G 66% 0
> /local/ds/xp27 200G 130G 68G 66% 0
>
That seems impossible.
Are you SURE you copy/pasted what I posted into files rather than retyping it? I
suspect your version of "myFile" is NOT the same one I posted.
Do this:
cat df_out
cat myFile
awk '{print FILENAME,$FNR,$0}' myFile df_out
and copy/paste the result into a posting so we can see what you're working with.
Ed.
| |
| spacegoose 2008-03-22, 6:58 pm |
|
> That seems impossible.
>
> Are you SURE you copy/pasted what I posted into files rather than retyping it? I
> suspect your version of "myFile" is NOT the same one I posted.
>
> Do this:
>
> cat df_out
> cat myFile
> awk '{print FILENAME,$FNR,$0}' myFile df_out
>
> and copy/paste the result into a posting so we can see what you're working with.
>
> Ed.
I am running Solaris 10. Also tried on Solaris 9 -same result.
xxx@x3$ cat df_out
/dev/md/ds/dsk/x20 200G 129G 69G 66% /local/ds/xp20
/dev/md/ds/dsk/x21 200G 135G 63G 69% /local/ds/xp21
/dev/md/ds/dsk/x22 200G 148G 50G 75% /local/ds/xp22
/dev/md/ds/dsk/x23 200G 128G 70G 65% /local/ds/xp23
/dev/md/ds/dsk/x24 200G 126G 72G 64% /local/ds/xp24
/dev/md/ds/dsk/x25 200G 136G 62G 69% /local/ds/xp25
/dev/md/ds/dsk/x26 200G 129G 69G 66% /local/ds/xp26
/dev/md/ds/dsk/x27 200G 130G 68G 66% /local/ds/xp27
xxx@x3$ cat myFile
11 621 2008/03/21 12:44 xp20 256000 user/foo/INBOX
806 41722 2008/03/21 13:26 xp21 256000 user/bar/INBOX
2391 115428 2008/03/21 13:26 xp22 256000 user/moo/INBOX
2452 122771 2008/03/21 12:38 xp23 256000 user/123/INBOX
638 65484 2008/03/21 12:56 xp23 256000 user/345/INBOX
139 4892 2008/03/21 12:44 xp24 256000 user/789/INBOX
398 13403 2008/03/19 14:05 xp24 256000 user/1234/INBOX
392 32448 2008/03/21 14:14 xp25 256000 user/5678/INBOX
xxx@x3$ awk '{print FILENAME,$FNR,$0}' myFile df_out
myFile 11 621 2008/03/21 12:44 xp20 256000 user/foo/
INBOX 11 621 2008/03/21 12:44 xp20 256000 user/foo/
INBOX
myFile 806 41722 2008/03/21 13:26 xp21 256000 user/bar/
INBOX 806 41722 2008/03/21 13:26 xp21 256000 user/bar/
INBOX
myFile 2391 115428 2008/03/21 13:26 xp22 256000 user/moo/
INBOX 2391 115428 2008/03/21 13:26 xp22 256000 user/moo/
INBOX
myFile 2452 122771 2008/03/21 12:38 xp23 256000 user/123/
INBOX 2452 122771 2008/03/21 12:38 xp23 256000 user/123/
INBOX
myFile 638 65484 2008/03/21 12:56 xp23 256000 user/345/
INBOX 638 65484 2008/03/21 12:56 xp23 256000 user/345/
INBOX
myFile 139 4892 2008/03/21 12:44 xp24 256000 user/789/
INBOX 139 4892 2008/03/21 12:44 xp24 256000 user/789/
INBOX
myFile 398 13403 2008/03/19 14:05 xp24 256000 user/1234/
INBOX 398 13403 2008/03/19 14:05 xp24 256000 user/1234/
INBOX
myFile 392 32448 2008/03/21 14:14 xp25 256000 user/5678/
INBOX 392 32448 2008/03/21 14:14 xp25 256000 user/5678/
INBOX
myFile
df_out /dev/md/ds/dsk/x20 200G 129G 69G 66% /local/ds/
xp20 /dev/md/ds/dsk/x20 200G 129G 69G 66% /local/ds/xp20
df_out /dev/md/ds/dsk/x21 200G 135G 63G 69% /local/ds/
xp21 /dev/md/ds/dsk/x21 200G 135G 63G 69% /local/ds/xp21
df_out /dev/md/ds/dsk/x22 200G 148G 50G 75% /local/ds/
xp22 /dev/md/ds/dsk/x22 200G 148G 50G 75% /local/ds/xp22
df_out /dev/md/ds/dsk/x23 200G 128G 70G 65% /local/ds/
xp23 /dev/md/ds/dsk/x23 200G 128G 70G 65% /local/ds/xp23
df_out /dev/md/ds/dsk/x24 200G 126G 72G 64% /local/ds/
xp24 /dev/md/ds/dsk/x24 200G 126G 72G 64% /local/ds/xp24
df_out /dev/md/ds/dsk/x25 200G 136G 62G 69% /local/ds/
xp25 /dev/md/ds/dsk/x25 200G 136G 62G 69% /local/ds/xp25
df_out /dev/md/ds/dsk/x26 200G 129G 69G 66% /local/ds/
xp26 /dev/md/ds/dsk/x26 200G 129G 69G 66% /local/ds/xp26
df_out /dev/md/ds/dsk/x27 200G 130G 68G 66% /local/ds/
xp27 /dev/md/ds/dsk/x27 200G 130G 68G 66% /local/ds/xp27
also:
xxx@x3$ more tst.awk
BEGIN {
printf "%-15s%12s%10s%10s%10s%10s\n", "File System", "bytes", "used",
"avail", "capacity", "accounts"
print
"-------------------------------------------------------------------"
}
NR==FNR{ accts["/local/ds/"$5]++; next }
/dsk/ { printf "%-15s%12s%10s%10s%10s%10s\n", $6, $2, $3, $4, $5,
accts[$6]+0 }
Thanks,
sg
| |
| Ed Morton 2008-03-22, 6:58 pm |
| On 3/22/2008 9:37 AM, spacegoose wrote:
> I am running Solaris 10. Also tried on Solaris 9 -same result.
Ahh, you're using old, broken awk (/usr/bin/awk on Solaris). Use a modern awk,
e.g. GNU awk (gawk), New awk (nawk) or /usr/xpg4/bin/awk on Solaris.
Ed.
| |
| Kenny McCormack 2008-03-22, 6:58 pm |
| In article <47E59349.30308@lsupcaemnt.com>,
Ed Morton <morton@lsupcaemnt.com> wrote:
>On 3/22/2008 9:37 AM, spacegoose wrote:
>
>Ahh, you're using old, broken awk (/usr/bin/awk on Solaris). Use a modern awk,
>e.g. GNU awk (gawk), New awk (nawk) or /usr/xpg4/bin/awk on Solaris.
>
> Ed.
>
This thread went on for quite a while, didn't it?
Amazing how many of these back-and-forths eventually boil down to:
Oh, by the way, I'm using Solaris.
(And if not that, the other one is: Oh, I'm writing the code on a
Windows box [and running/testing it on Unix])
| |
| Ed Morton 2008-03-22, 10:01 pm |
|
On 3/22/2008 6:23 PM, Kenny McCormack wrote:
> In article <47E59349.30308@lsupcaemnt.com>,
> Ed Morton <morton@lsupcaemnt.com> wrote:
>
>
>
> This thread went on for quite a while, didn't it?
>
> Amazing how many of these back-and-forths eventually boil down to:
> Oh, by the way, I'm using Solaris.
>
> (And if not that, the other one is: Oh, I'm writing the code on a
> Windows box [and running/testing it on Unix])
>
The other biggy is "post sample input and expected output". I always forget to
say "post sample input and THE expected output FROM THAT INPUT". Got to remember
that....
Ed.
| |
| spacegoose 2008-03-23, 4:00 am |
| On Mar 22, 7:44 pm, Ed Morton <mor...@lsupcaemnt.com> wrote:
> On 3/22/2008 6:23 PM, Kenny McCormack wrote:
>
>
>
>
>
>
>
>
>
>
>
> The other biggy is "post sample input and expected output". I always forget to
> say "post sample input and THE expected output FROM THAT INPUT". Got to remember
> that....
>
> Ed.
That did it. Thanks. One last thing: Can this be combined into one
file to act on myFile:
so that everything is contained in one file (with a reference to
myFile inside)?
| |
| Ed Morton 2008-03-23, 4:00 am |
| On 3/23/2008 12:13 AM, spacegoose wrote:
<snip>
> That did it. Thanks. One last thing: Can this be combined into one
> file to act on myFile:
> so that everything is contained in one file (with a reference to
> myFile inside)?
I think what you want is to put this in a file:
------------
df -h | sort +5 | awk '
BEGIN {
printf "%-15s%12s%10s%10s%10s%10s\n", "File System", "bytes", "used", "avail",
"capacity", "accounts"
print "-------------------------------------------------------------------"
}
NR==FNR{ accts["/local/ds/"$5]++; next }
/dsk/ { printf "%-15s%12s%10s%10s%10s%10s\n", $6, $2, $3, $4, $5, accts[$6]+0 }
' myFile -
-------------
Regards,
Ed.
| |
| Rajan 2008-03-23, 10:00 pm |
| you can do this with a combination of sort and paste but if you are a gawky
like me
BEGIN{
cmd="df -h | grep whatever | sort -n +5"
while(cmd | getline)
{ mntspace[$1]=$2;mntused[$1]=$3}
cmd="awk {'print $5'} myFile |sort |uniq -c |sort -nr"
while(cmd|getline)
{mntacs[$2]=$1}
printf "%-12s %-12s %-12s %-12s", "Disk", "Space", "Used", "Accounts"
for (each1 in mntspace) {printf "%-12s %-12s %-12s %-12s", each1,
mntspace[each1], mntused[each1], mntacs[each1]}
exit
}
"spacegoose" <spacegoose@gmail.com> wrote in message
news:8acca503-5eaf-4d95-abca-c8fd6390f99c@o22g2000hsh.googlegroups.com...
> i have a program that prints out a formatted df command
> by sorting and grepping and using awk to print it out.
>
> it basically executes df -h | grep whatever | sort -n +5 and then
> awks
> to a nice printout like:
>
> disk size capacity
> ------------------
> foo 20gb 70%
> moo 40gb 25%
> bar 20gb 70%
>
> i'd like to add a new column "accounts" to this print out.
>
> the no. of accounts is derived from a command:
> awk {'print $5'} myFile |sort |uniq -c |sort -nr
>
> which prints rows like:
>
> 450 foo
> 300 moo
> 104 bar
>
> i want to integrate this command into the formatted df program's
> output.
> i can't figure out how to appropriately append the "account count"
> column from the myFile command, to the appropriate row of the
> formatted df output,
> e.g. where col 1 from the formatted df output matches col 2 of the
> myFile command.
>
> so it looks like this:
>
>
> disk size capacity accounts
> ---------------------------
> foo 20gb 70% 450
> moo 40gb 25% 300
> bar 20gb 70% 104
>
>
> Thanks for any hints!
> sg
| |
| Ed Morton 2008-03-23, 10:00 pm |
| On 3/23/2008 4:04 PM, Rajan wrote:
[please don't top-post, fixed below]
> "spacegoose" <spacegoose@gmail.com> wrote in message
> news:8acca503-5eaf-4d95-abca-c8fd6390f99c@o22g2000hsh.googlegroups.com...
>
>
> you can do this with a combination of sort and paste but if you are a gawky
> like me
>
> BEGIN{
> cmd="df -h | grep whatever | sort -n +5"
> while(cmd | getline)
> { mntspace[$1]=$2;mntused[$1]=$3}
> cmd="awk {'print $5'} myFile |sort |uniq -c |sort -nr"
> while(cmd|getline)
> {mntacs[$2]=$1}
> printf "%-12s %-12s %-12s %-12s", "Disk", "Space", "Used", "Accounts"
> for (each1 in mntspace) {printf "%-12s %-12s %-12s %-12s", each1,
> mntspace[each1], mntused[each1], mntacs[each1]}
> exit
> }
>
I don't mean to be rude, but the above is missing the point of awk as a
text-processing tool and trying to force it to behave like a shell, i.e. an
environment for invoking tools. It's totally the wrong approach to solving this
or any other problem.
Ed.
| |
| Kenny McCormack 2008-03-23, 10:00 pm |
| In article <47E6D0B5.4010608@lsupcaemnt.com>,
Ed Morton <morton@lsupcaemnt.com> wrote:
....
>I don't mean to be rude, but the above is missing the point of awk as a
>text-processing tool and trying to force it to behave like a shell, i.e. an
>environment for invoking tools. It's totally the wrong approach to solving this
>or any other problem.
>
> Ed.
>
>
That's what makes it so  .
| |
| Rajan 2008-03-23, 10:00 pm |
| Totally agreed. I'm being misunderstood, this is definitely not about the
filter this is just about matching the multiple columns. I should have
mentioned that in my post!
Rajan
"Ed Morton" <morton@lsupcaemnt.com> wrote in message
news:47E6D0B5.4010608@lsupcaemnt.com...
> On 3/23/2008 4:04 PM, Rajan wrote:
>
> [please don't top-post, fixed below]
>
>
> I don't mean to be rude, but the above is missing the point of awk as a
> text-processing tool and trying to force it to behave like a shell, i.e.
> an
> environment for invoking tools. It's totally the wrong approach to solving
> this
> or any other problem.
>
> Ed.
>
>
| |
| spacegoose 2008-03-24, 6:59 pm |
| On Mar 23, 1:41 am, Ed Morton <mor...@lsupcaemnt.com> wrote:
> On 3/23/2008 12:13 AM, spacegoose wrote:
> <snip>
>
>
> I think what you want is to put this in a file:
>
> ------------
> df -h | sort +5 | awk '
> BEGIN {
> printf "%-15s%12s%10s%10s%10s%10s\n", "File System", "bytes", "used", "avail",
> "capacity", "accounts"
> print "-------------------------------------------------------------------"}
>
> NR==FNR{ accts["/local/ds/"$5]++; next }
> /dsk/ { printf "%-15s%12s%10s%10s%10s%10s\n", $6, $2, $3, $4, $5, accts[$6]+0 }
> ' myFile -
> -------------
>
> Regards,
>
> Ed.
This is working, Thanks Ed.
Can you make the columns tally at the bottom?
| |
| Ed Morton 2008-03-26, 7:03 pm |
|
On 3/24/2008 10:55 AM, spacegoose wrote:
> On Mar 23, 1:41 am, Ed Morton <mor...@lsupcaemnt.com> wrote:
>
>
>
> This is working, Thanks Ed.
> Can you make the columns tally at the bottom?
df -h | sort +5 | awk '
BEGIN {
printf "%-15s%12s%10s%10s%10s%10s\n", "File System", "bytes", "used", "avail",
"capacity", "accounts"
print "-------------------------------------------------------------------"
}
NR==FNR{ accts["/local/ds/"$5]++; next }
/dsk/ {
printf "%-15s%12s%10s%10s%10s%10s\n", $6, $2, $3, $4, $5, accts[$6]+0
tot["bytes"]+=$2
tot["used"]+=$3
tot["avail"]+=$4
tot["capacity"]+=$5
tot["accounts"]+=accts[$6]
}
END {
printf "%-15s%12s%10s%10s%10s%10s\n", "Totals", tot["bytes"], tot["used"],
tot["avail"], tot["capacity"], tot["accounts"]
}
' myFile -
|
|
|
|
|