[PLUG] Capture of CSV data

Rich Shepard rshepard at appl-ecosys.com
Mon Jun 23 21:30:33 UTC 2014


On Mon, 23 Jun 2014, Chuck Hast wrote:

> Hmmm I guess I did not get the data sample in it.

Chuck,

   You did include the data sample, but not what questions you had about it.

> I know how to get csv files into spead sheets and i have sort of a idea of
> getting them into a db, but how do I handle a stream of data like that
> below. All of them are CSV formated, so once I can get one into something
> I can graph and twiddle, the rest should be easy.

   So, you want to store each row in a database table. Then you want to
produce some sort of plot (histogram? scatter plot? box-and-whisker plot?)
from selected rows or continuously? What sort of twiddling do you envision?

   I would approach a solution by using a Python script (the psycopg2 module
is appropriate for the interface between python and postgres, and wxPython
is the GUI I use) to store the rows in a postgres database table as each row
was sent from a machine, then use pandas to analyze those data and produce
plots using matplotlib. There are probably a gazillion alternatives,
including using R for the analyses, but I suspect that would be overkill for
your needs.

Rich




More information about the PLUG mailing list