[PLUG] removing dups

[Matt Alexander]

On Thu, 8 Apr 2004, alan wrote:

> On Thu, 8 Apr 2004, Matt Alexander wrote:
> 
> > I have a tab-delimited text file that I'm inserting into a database.  The
> > first three fields need to be unique.  Sometimes the text file has
> > duplication of the first three fields, but the rest is different, so I
> > can't simply use "sort -u" to remove the dups.  The first occurance of the
> > dupe is always the one that I want removed.
> > 
> > Any suggestions on a way to find the dups and remove the first occurance?
> > Thanks,
> 
> Do a select based on those three fields.  If you get a "hit", replace the 
> record with the current one.  If you don't, insert the line as a new 
> record.

The import is actually failing because those three fields comprise the
primary key.  I'll look into the Perl solution that Colin suggested.
Thanks,
~M