Data Hosting

The RBRcervello is optionally supplied with real-time telemetry in the form of GSM cell and Iridium satellite modems. These send data to the RBR data hosting service, which is accessible at https://data.rbr-global.com/ClientReference. Each client site may be public or password protected. One-year of data hosting service is included with each RBRcervello that includes telemetry. Additional years of data hosting service can be purchased following the first year of ownership. 

Obtaining CSV data files

The following information can be used to obtain data in the form of comma-separated value (CSV) files from the RBR data hosting portal.

A note about RBRcervello

Data controllers are a slightly special case because they report data under two different serial numbers: one for the attached instrument (RBRconcerto/RBRmaestro), and one for diagnostic information (battery levels and GPS coordinates). Remember to download from the appropriate instrument.

Obtain data from the RBR data hosting website

The simplest method for downloading files for an instrument is to navigate to https://data.rbr-global.com/rbr (substitute RBR with your customer name) and use the download button.

Obtaining data from RBR data hosting using a shell script

Data from a particular instrument can be periodically polled and downloaded by using the getcsvdata.sh script. This script downloads data for a particular instrument from the data hosting portal operated by RBR. After downloading the script, you will have to change the file permissions to enable execution. For example, to permit all users on the system to execute the script:

Set Script Permission
chmod +x getcsvdata.sh

Call the script using your customer URL slug, serial number, start date, and end date as arguments. For example, to download all data for RBR's device '110099' from September 1, 2017, you can use the script with this command:

Get Data Example
./getcsvdata.sh rbr 110099 2017-09-01 2020-12-31

Manually download data files

In addition to the above methods of acquiring CSV files, exports of instrument data can be retrieved by making HTTP GET requests following this URL pattern:


https://data.rbr-global.com/customer/download/serial?from=A&to=B

or

https://data.rbr-global.com/customer/download/serial?timespan=C

  • customer is the customer slug; e.g. rbr, csiro
  • serial is the serial number of the instrument for which data should be retrieved
  • A and B are dates in YYYY-MM-DD format and respect the instrument timezone.
    • A and B can also contain the time of day in YYYY-MM-DDTHH:MM:SS.
  • C is an integer number of days

The first format will yield a CSV of all data from dates A to B inclusive of both, except when the time of day is specified. The second format will yield a CSV of all data from the last C × 24 hours (not the most recent C × 24 hours of data available!).

For example, the above URL format could be used within a 'curl' command to obtain the data from RBR's instrument with serial number 080296 from September 8, 2016 to September 9, 2016:

Example download using 'curl'
curl --location --remote-name --remote-header-name --compressed 'http://data.rbr-global.com/rbr/download/080296?from=2016-09-08&to=2016-09-09'

As another example, the CSV could also be requested with data from September 9, 2016, at 11:00:00 to 22:00:00:

Example download using 'curl'
curl --location --remote-name --remote-header-name --compressed 'http://data.rbr-global.com/rbr/download/080296?from=2016-09-09T11:00:00&to=2016-09-09T22:00:00'

Remember to enclose the URL in quotation marks, lest your shell split the command around the ampersand (&) character.

In general, you should restrict yourself to downloading around one or two days of data at a time, and later concatenating the results. An instrument sampling at 8Hz will generate a little more than 100MB of CSV data per day, leading to a distinct possibility that larger transfers may time out during the download process, the connection may be unexpectedly dropped, etc.

Low bandwidth transfers

For circumstances where bandwidth access is limited the following workflow could be used. The getcsvdata.sh script described above can be used to acquire data and save to a server that has a high bandwidth always-on connection. Subsequent transfer to a computer with relatively low bandwidth can be accomplished by transferring only the difference using the 'rsync' command.

Use 'rsync' for data pull from low bandwidth remote computers
rsync --verbose --archive --partial --progress --compress myusername@my.server.address:/my/data/path/RBR-080296.csv .

Downloads

getcsvdata.sh