I am reading in from a MySQL database and writing out to a CSV using a "|" character as a separator. Some columns in my DB have a "|" as part of their value so my CSV output very often get messed up. Is there any way to run all columns values through a component to remove all "|" before I attempt to write the output to CSV? I know I could check every single metadata with a Reformat component but I would have to write that for literally 100s of columns, not something I am excited to do. Suggestions welcome.
Thanks!
Thanks!
-
Some time ago, Paulhbartosik posted a generic solution for trimming strings. With small change, you can tweak it for your purpose: http://forum.cloveretl.com/viewtopic.php?f=7&t=7545.
But, may I suggest to enable "Quoted strings" in your FlatFileWriter? That way, you should not need to remove them at all. Some people are using CSV files as "stage" area in which case I'd recommend using CloverDataWriter instead - that will create semi-binary file containing both metadata and data. Benefit is, reading of this file is much faster than parsing it using FlatFileReader.
Cheers! -
Hi Jesse,
I replied to a similar question on how to replace on all fields without having to go one by one. I used the reformat and a code I created to accomplish this:
http://forum.cloveretl.com/viewtopic.php?f=4&t=7306&hilit=replace#p12426
Please sign in to leave a comment.
Comments 2