[PLUG] An awk-ward Question

[Rich Shepard]

On Tue, 22 Mar 2011, Fred James wrote:

> However ... since you are working with SQL to extract the data in the
> first place, you could just format the data as required and skip the AWK
> step altogether, could you not?

Fred,

   Nope. I did not clearly explain. I have dozens of Excel spreadsheets with
a total of about 17 years worth of data. There were/are no standards for
organizing each sheet, what data are placed where, the names of parameters
and units, or anything else. Total data anarchy; a horrible mess that
requires me to wear hip waders as I go through the process of taking each
sheet (first saved as an .ods file) to .csv then input into a postgres
table.

   The SQL is to check that there are no attribute inconsistencies, check my
progress (45.5K rows so far), and make sure everything is working.

Rich