ChrisPope (Customer) asked a question.

CSV Identity Collector throwing BIGINT error

I am modifying one of our two IDCs to accommodate the change from our old Contractor Identity Source to the new one. We are using a "final file" .csv from the old source in a LEFT JOIN to the new .csv so that user being migrated from old to new are not deleted/re-created.

 

The field being used as the UserID in IGL from the old source is numeric whereas the field from the new source is alphanumeric (beginning with "WO"). When I test the query on the "Mapping for user attributes" page, it throws this error (NOTE: I've replaced sensitive info with asterisks (****):

 

This statement type might not be supported, or it has syntax errors: java.sql.SQLException: Failed to go Record 1497 of table *****_Contractors:Value WO*****(row 1497, column Engt ID) is not BIGINT type

 

These are the settings for the connection:

?_CSV_Header=true;_CSV_Quoter=";maxScanRows=20000;delayedClose=0

imageFrom what I understand, the driver being used (com.hxtt.sql.text.TextDriver) interprets the data-type by scanning the data and has determined that the UserID is an integer. This causes the above error because the data from the new Identity Source begins with "WO".

 

I have tried several workarounds to force the driver to view it as alphanumeric, including putting the "WO" records first but nothing seems to work. Perhaps it uses the majority of the data to determine the data-type, and since the alphanumeric records (WO) are very small, it doesn't take them into consideration.

 

I also tried maxScanRows=0:

?_CSV_Header=true;_CSV_Quoter=";maxScanRows=0;delayedClose=0

 

 Any suggestions or help would be appreciated.

 

-Chris


  • ChrisPope (Customer)

    I figured out a solution with the help of AI. I created a schema.ini file which I placed on the server in the same directory as the two .csv input files being collected. I defined the data-types for each of the columns, which forced the driver to see the two columns being used in the LEFT JOIN as being "Text". This allowed the two files to be collected.