Hi,
I found some posts about using Split and File Format Convert to split files. I'd like to be able to split CR3000 *.dat files by month. I'm currently doing this is Python with the datetime and csv modules and outputting to CSV.
I figured out how to use Split to get that timestamp range right
e.g.
Start Condition = 1:[2017]:1[3%1]::1
Stop Condition = 1:[2017]:1[4%1]::1
to get March 2017, The problem I have with this is that it is very slow. (My data is every 5 minutes) and the headers are thrown away.
It seems that File Format convert would be the way to go, but all months do not have the same number of days. It does seem to be faster than Split.
Is there a way to do this with FFC, or should I just stick with my python script?
Thanks.
There are a couple of options you can consider.
I hope one of these will help!
Best, Dana
Dana,
Thanks for the suggestions. I think the last option using Task Master sounds like the best option for me. I want to keep the headers in each file. Now if I'm collecting data every 5 minutes, should I schedule this at say 2357 at the end of each month, so I get the last collection of the month at 2355?
I'll also test out the other options, but I'm definitely leaning on the last option
Regards, Tim
Yes, I think collecting at some point between 2355 and the top of the hour should work (I also think this is your best option if the only reason you were using Split was to get a monthly file). Keep in mind that the downside to this option is if you have a failed data collection attempt, you could potentially end up with two file(s) that have more or less data than you expect. You'll still end up with all the data, but you might have to occassionally edit the files to get the right month into the right file.
Unless you have a problematic communication link, however, the likelihood of this happening is fairly low.
Dana