Hello,
we actually try to automise to the maximum data retrieval on all our dataloggers, vithout PC400 or loggernet, because there remains no linux version of these (yet), and our servers are linux.
But several problems comes during the last year :
1 - I make a program which use "TableFile" function to create one file per day, it's more like the rest of our research data. See below :
'CR300 Series
'Created by Short Cut (4.0)
'Declare Variables and Units
Public BattV
Public PTemp_C
Public TRHData(2)
Public WS_ms
Public WindDir
Public flux_density
Public total_flux
Alias TRHData(1)=AirTC
Alias TRHData(2)=RH
Const LoggerID="OGHV"
Public Outstat1 As Boolean
Public Outstat2 As Boolean
Public LastFileName1 As String * 20
Public LastFileName2 As String * 20
Public rTime(9)
Units BattV=Volts
Units PTemp_C=Deg C
Units WS_ms=meters/second
Units WindDir=degrees
Units flux_density=W/m^2
Units total_flux=kJ/m^2
Units AirTC=Deg C
Units RH=%
'Define Data Tables
DataTable(meteo,True,-1)
DataInterval(0,10,Min,10)
Average(1,AirTC,FP2,False)
Sample(1,RH,FP2)
Average(1,WS_ms,FP2,False)
Maximum(1,WS_ms,FP2,False,True)
StdDev(1,WS_ms,FP2,False)
Sample(1,WindDir,FP2)
Average(1,flux_density,FP2,False)
Maximum(1,flux_density,FP2,False,True)
StdDev(1,flux_density,FP2,False)
Totalize(1,total_flux,IEEE4,False)
TableFile ("CPU:meteo",8,-1,0,24,Hr,Outstat1,LastFileName1)
EndTable
DataTable(soh,True,-1)
DataInterval(0,60,Min,10)
Minimum(1,BattV,FP2,False,False)
Average(1,PTemp_C,FP2,False)
TableFile ("CPU:soh",8,-1,0,24,Hr,Outstat2,LastFileName2)
EndTable
'Main Program
BeginProg
'Main Scan
Scan(60,Sec,1,0)
'Default CR300 Datalogger Battery Voltage measurement 'BattV'
Battery(BattV)
'Default CR300 Datalogger Processor Temperature measurement 'PTemp_C'
PanelTemp(PTemp_C,50)
'CS215 Temperature & Relative Humidity Sensor measurements 'AirTC' and 'RH'
SDI12Recorder(TRHData(),C1,"0","M!",1,0,-1)
'05103 Wind Speed & Direction Sensor (CSL) measurements 'WS_ms' and 'WindDir'
PulseCount(WS_ms,1,P_LL,1,1,0.098,0)
BrHalf(WindDir,1,mV2500,1,VX1,1,2500,False,20000,50,355,0)
If WindDir>=355 Or WindDir<0 Then WindDir=0
'CMP3/CMP6/CMP11 Pyranometer (CSL) measurements 'total_flux' and 'flux_density'
VoltDiff(flux_density,1,mV34,2,True,0,50,1,0)
If flux_density<0 Then flux_density=0
total_flux=flux_density*12
flux_density=flux_density*200
'Call Data Tables and Store Data
CallTable meteo
CallTable soh
If Outstat1 Then
FileRename(LastFileName1,"CPU:"+rTime(1)+"_"+rTime(9)+"_"+rTime(4)+"_"+rTime(5)+"_meteo_"+LoggerID+".dat")
EndIf
If Outstat2 Then
FileRename(LastFileName2,"CPU:"+rTime(1)+"_"+rTime(9)+"_"+rTime(4)+"_"+rTime(5)+"_soh_"+LoggerID+".dat")
EndIf
NextScan
SlowSequence
Scan (60,Sec,3,0)
NetworkTimeProtocol ("51.255.197.148",0,100) 'ntp.pool...
RealTime(rTime)
NextScan
EndSequence
EndProg
First problem, sometimes the file was created at 00:00, and sometimes at 00:01 : somebody know why ?
Second problem, after a year of working fine, the CR310 stop create file everyday. No one more. But it still record in these data table. I checked the free memory : >80Mb. I had to erase all data, stop the program, erase the program, and send it back to run again and the day file have been be create again. Any idea on it ?
2 - We get the files remotely by using the FTP server on the CR310, and use "lftp" function on linux. It copy all CPU folder on a remote folder (CR310 can't have dedicated USR partition).
Third problem, FTP server had unchecked his boxe in the deployment option visible in DevConfig (?), but nevermind.
Fourth problem, more a question. CR1000 could be requested to send some time stamped file via WGET. CR310 can too ? Linux command are something like :
"${IP_STA}/?command=dataquery&uri=dl:${TABLE}&format=toa5&mode=since-record&p1=${LAST_RECORD}"
(Where IP_STA is an ip address and LAST_RECORD is a date. I think)
Thanks for your help,
bye
Hi, there is a LoggerNet Linux version available. See details here :-
If you have a large number of files on the CPU drive, file access will slow down a lot.
Hello,
well, thanks for your answers.
nsw : unfortunatly our servers doesn't have graphical interface, there are only data retrieval node and super calculator. They are linux based but there is no desk, and so no graphical loggernet possible.
JDavis : ok, i get it. But do you think in my prog this is explain that no one files were created during several weeks ? Last file was created 28th january, I see this problem few days ago. And so, can you give me a program command line to automatically delete files when they are more than X files (typically 30 files, for example), please (if it's possible, of course) ?
What do you think about the hour of the file creation ? 00:00 and 00:01 appear even when there are one or two files, and disappear, and come back, etc.
And more important, do you know if "wget" is possible on CR310 ?
Thanks for your help,
have a nice day.
Hello,
to give some news :
Problem 1 : file creation, the time stamp can be 00:00 or 00:01, randomly, don't know why. It still a problem but not critical at all.
Problem 2 : still don't know what's happened, JDavis give me maybe a clue because of the number of file : can I program a auto-delete files function (older than 30 days for exemple) ?
Problem 3 : Maybe something, a CR310 in OS 06 continue to stop FTP server automatically, I have to use DevConfig to uncheck ftp box, and check it again. Then it works (for few days). A CR310 in 08 OS version, don't have this problem. When we will go on the field again, I'll proceed to an update, maybe that's the problem source.
Problem 4 : the wget. We can make a wget through the web server (port 80). But there is a problem to display all information in the page. On the main page "status" I have all the label but not the value, and at the bottom of the page there is :
ERROR: undefined
DISCONNECTED
However I can read some files, but not all. Strange. I can be the OS version too, difficult to say.
What do you think about it ?
Thanks, bye
Hello,
the problem seem to be cyclic, after 2 - 3 days working fine, the FTP server stop.
The port 21 still online (ping), but serveur is timeout.
I have to connect, deactivate FTP, reactivate, and it's work again.
If you are updating the OS on the datalogger, you should be using OS 10.01. It uses a newer version of the IP stack than OS 8.
Regarding Problem 1: OutStat is set true when the file writing process has been completed and the file is closed. The writing of the file, triggered by the calling of TableFile, is a background process. The program does not wait for the file writing process to complete. CallTable executes, the records are written to flash, and TableFile kicks off a background process to write the file. This may take longer than the amount of time it takes to return to the IF statement used to evaluate OutStat. As such, OutStat = TRUE isn't realized until the subsequent SCAN, occuring 60 seconds / 1 minute later. When that happens, the code as written will set the file name with MINUTES = 1. One solution is to delay or wait a maximum amount of time for OutStat to go TRUE during the same scan. Another solution is to use the FileTime of the file when forming up the filename, ignoring resolution finer than 1 minute.
Regarding 2: Writing of the files should not stop as long as there is sufficient room to write the file. It would be great if you cold work with your local Campbell Scientific office to run this behavior to ground. If it is a bug, we would like to address it. Your FTP client can collect and delete files matching the "*.dat" name pattern. Alternatively, you can use FileList and FileManage to delete old files.
Also, we offer LNLinux, and LNLinux can be administered via command line on the Linux box AND/OR with LN or LN Remote from a Windows box, remotely connected.
Regarding 3: It would be great if you cold work with your local Campbell Scientific office to run this behavior to ground. If it is a bug, we would like to address it.
Regarding 4: The CR300 has the same HTTP API as the CR1000.
Hi,
to "feed" the topic, here is my new strategy. As I'm better in bash programmation, than CRBasic, I decided to let the campbell stuff record more "normally", not creating a file by day, but just have data in table as usual.
And I made a bash script that's make request on the API, and make the files on our server.
The script read what's on the datalogger, compare with what is on our server, and make all the files are needed to be created since the last communication.
I'm not authorize yet to go on the field, but I was able to deploy test stuff in my lab, so I have a CR1000 and a CR300 online to test my script.
Thanks for help,
bye