Difference between revisions of "PRO History"
| (10 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
| − | |||
__NOTOC__ | __NOTOC__ | ||
| − | + | One of the features that sets Meteobridge PRO apart from the regular Meteobridges is the ability to store 1 GB (2 GB in case of NANO SD) of sensor data on local storage. Being able to store data locally is the foundation for additional services. | |
Meteobridge PRO and NANO SD store data short term (last two days) in per-minute intervals and long-term in hourly intervals and automatically aggregate hourly data update to daily, monthly and yearly levels. By that Meteobridge PRO can provide information on different time resolution levels in no time. Meteobridge PRO takes care that data of different time resolution levels are always in perfect sync. Data for non-cumulative units is stored as average, minimum and maximum for each of the time resolutions. Cumulative units (like rain totals) just provide summed-up data. Beside min and max values also time stamps of min and max events are stored. | Meteobridge PRO and NANO SD store data short term (last two days) in per-minute intervals and long-term in hourly intervals and automatically aggregate hourly data update to daily, monthly and yearly levels. By that Meteobridge PRO can provide information on different time resolution levels in no time. Meteobridge PRO takes care that data of different time resolution levels are always in perfect sync. Data for non-cumulative units is stored as average, minimum and maximum for each of the time resolutions. Cumulative units (like rain totals) just provide summed-up data. Beside min and max values also time stamps of min and max events are stored. | ||
| Line 18: | Line 17: | ||
'''Month''' shows all months of the selected year with the ones with sensor data marked with a "*". In the example below month November is selected and graph on the right displays all days of the month. | '''Month''' shows all months of the selected year with the ones with sensor data marked with a "*". In the example below month November is selected and graph on the right displays all days of the month. | ||
| − | [[File:page7b.png]] | + | [[File:page7b.png|685px]] |
'''Day''' selects a day in the month and summarizes all hours of the day in the graph on the right. | '''Day''' selects a day in the month and summarizes all hours of the day in the graph on the right. | ||
| − | [[File:page7c.png]] | + | [[File:page7c.png|685px]] |
Display of values can be switched to imperial units with the "use Imperial Units" check box in the head of the "Edit Data" field. | Display of values can be switched to imperial units with the "use Imperial Units" check box in the head of the "Edit Data" field. | ||
| − | ===Editing Data of an | + | ===Editing Data of an Hour=== |
| − | A click onto | + | A click onto an hour time stamp starts deletion of the sensor data for that hour, when you confirm the pop-up dialog to do so. |
| − | A click onto the average, minimum or maximum values opens an input dialog, where a new value can be manually inserted. When no units are given with the number it is assumed that these are metric units. If you want to insert data with imperial units, | + | A click onto the average, minimum or maximum values opens an input dialog, where a new value can be manually inserted. When no units are given with the number it is assumed that these are metric units. If you want to insert data with imperial units, please add "F", "in", "mph", "inhg" to your values. A field that shows "--" indicates that the is no sensor data available for this minute. When data repeats across average, minimum, maximum the min/max fields are marked with a double quote to avoid visual overloading by repeating data. |
| − | [[File:page7d.png]] | + | [[File:page7d.png|685px]] |
===Quick Relative Navigation=== | ===Quick Relative Navigation=== | ||
| − | * '''Now''' button directs to the current | + | * '''Now''' button directs to the current day and shows data of that hour for the selected sensor. |
| − | * '''< <''' button skips to the past. If drill-down level is on | + | * '''< <''' button skips to the past. If drill-down level is on daily level, previous day is selected. When drill-down level is months, previous month is selected, and so on. |
* '''> >''' button skips one step to the future using the current drill-down level. | * '''> >''' button skips one step to the future using the current drill-down level. | ||
| Line 42: | Line 41: | ||
* '''Delete All''' button deletes data for all sensors. | * '''Delete All''' button deletes data for all sensors. | ||
Before the delete operations are performed you are asked to confirm deletion. When deletion is done, data cannot be recovered. | Before the delete operations are performed you are asked to confirm deletion. When deletion is done, data cannot be recovered. | ||
| + | |||
| + | ==Import Data== | ||
| + | Meteobridge PRO and NANO SD can inport data from various sources, including | ||
| + | * Meteohub monthly raw data | ||
| + | * Meteoplug monthly esport data | ||
| + | * Meteotemplate | ||
| + | * Cumulus | ||
| + | * weewx | ||
| + | * WSWIN | ||
| + | Import operations can take some time as thousands of data sets need to be stored in Meteobridge's internal database. In order to import a file it needs to be dropped into the "import" folder of Meteobridge's PC-network share. Import can be told to stop on errors or to ignore those. Import operation is logged in messages section of "System" tab. | ||
| + | |||
| + | [[File:page7e.png|685px]] | ||
==Export Data== | ==Export Data== | ||
| − | Meteobridge PRO's PC network folder contains a "templates" sub folder where export templates are stored. Template drop-down menu provides predefined standard templates for data export but will also present user-defined export templates (files with extension ".exp"). | + | Meteobridge PRO's or NANO SD's PC network folder contains a "templates" sub folder where export templates are stored. Template drop-down menu provides predefined standard templates for data export but will also present user-defined export templates (files with extension ".exp"). |
===Export Template Format=== | ===Export Template Format=== | ||
| Line 60: | Line 71: | ||
Meteobridge PRO also defines pragmas, that control how special data situations are handled. Pragmas are lines that start with "#pragma" and contain some of the following control settings: | Meteobridge PRO also defines pragmas, that control how special data situations are handled. Pragmas are lines that start with "#pragma" and contain some of the following control settings: | ||
| − | * '''emptylines= | + | * '''emptylines=n''' blocks lines that don't have any sensor data from being printed into the target file. Default is that empty lines are printed (emptylines=y). |
| + | * '''futuredata=n''' blocks lines that have time stamps in the future. Default is futuredata=n. | ||
| + | * '''crlf=y''' provides Windows style line ends (CR + LF) instead of Linux style (LF only). Default is crlf=n. | ||
| + | * '''log=y''' logs data export execution in Meteobridsge system messages. Default is log=y. | ||
| + | * '''fillgap=120''' gives a time period data of a sensor is assumed to be recent. During that period the data of sensors is repeated into following lines of export data in case the sensor has not provided fresh data. Default is 600 seccond (10 minutes). | ||
| + | * '''exportfile=myexport.txt''' specifies a dedicated name of the export file. By default the name of the template and the chosen time period is used to generate a file name automatically. | ||
===Starting Exports=== | ===Starting Exports=== | ||
| − | Depending on the drill-down in the "Inspect Stored Data" section "Export Data" field presents export options ranging from exporting data of | + | Depending on the drill-down in the "Inspect Stored Data" section "Export Data" field presents export options ranging from exporting data of a day to data of a whole year. This exports can be done in resolution of a minute, hour, day or month. Combination of export of a whole year's data and resolution down to a minute is not allowed as this will produce too much data. All other combinations can be done. |
Exported data is stored on the PC network folder in subfolder "export" and is named accordingly to the used template, the time range and the used time resolution. To give an example, "standard-iso-201511-day.txt" is an export generated by export template "standard-iso", it is data from November 2015 and data is provided in a day resolution. Data in the file might look like this: | Exported data is stored on the PC network folder in subfolder "export" and is named accordingly to the used template, the time range and the used time resolution. To give an example, "standard-iso-201511-day.txt" is an export generated by export template "standard-iso", it is data from November 2015 and data is provided in a day resolution. Data in the file might look like this: | ||
Latest revision as of 15:04, 17 January 2021
One of the features that sets Meteobridge PRO apart from the regular Meteobridges is the ability to store 1 GB (2 GB in case of NANO SD) of sensor data on local storage. Being able to store data locally is the foundation for additional services.
Meteobridge PRO and NANO SD store data short term (last two days) in per-minute intervals and long-term in hourly intervals and automatically aggregate hourly data update to daily, monthly and yearly levels. By that Meteobridge PRO can provide information on different time resolution levels in no time. Meteobridge PRO takes care that data of different time resolution levels are always in perfect sync. Data for non-cumulative units is stored as average, minimum and maximum for each of the time resolutions. Cumulative units (like rain totals) just provide summed-up data. Beside min and max values also time stamps of min and max events are stored.
Foundation for making use of stored data is that there are management functions available that allow to easily inspect, edit and cleanup data. Picture below shows Meteobridge PRO "History" tab that is the data management console of Meteobridge PRO.
Inspect Data
Inspect data field allows you to select a piece of sensor data you are interested in and to drill down by selecting year, month, days to the hourly data that then can be changed, inserted or deleted.
Sensor allows to select the sensor to inspect. Without selecting a sensor a further drill-down is not possible.
Year shows all years beginning at 2014. When a year contains data (for the selected sensor) it is marked with an asterisk "*". When a year is selected the graph on the right does show all months of the year and the average, min and max values of that specific sensor for the year. If it is a cumulative sensor (like rain totals) only the total sum is presented. Example below shows 2015 with data for th0temp in months November and December.
Month shows all months of the selected year with the ones with sensor data marked with a "*". In the example below month November is selected and graph on the right displays all days of the month.
Day selects a day in the month and summarizes all hours of the day in the graph on the right.
Display of values can be switched to imperial units with the "use Imperial Units" check box in the head of the "Edit Data" field.
Editing Data of an Hour
A click onto an hour time stamp starts deletion of the sensor data for that hour, when you confirm the pop-up dialog to do so. A click onto the average, minimum or maximum values opens an input dialog, where a new value can be manually inserted. When no units are given with the number it is assumed that these are metric units. If you want to insert data with imperial units, please add "F", "in", "mph", "inhg" to your values. A field that shows "--" indicates that the is no sensor data available for this minute. When data repeats across average, minimum, maximum the min/max fields are marked with a double quote to avoid visual overloading by repeating data.
- Now button directs to the current day and shows data of that hour for the selected sensor.
- < < button skips to the past. If drill-down level is on daily level, previous day is selected. When drill-down level is months, previous month is selected, and so on.
- > > button skips one step to the future using the current drill-down level.
Data Deletion
Depending on the drill-down level data for an hour, day or month can be deleted.
- Delete One button deletes data for the selected sensor only.
- Delete All button deletes data for all sensors.
Before the delete operations are performed you are asked to confirm deletion. When deletion is done, data cannot be recovered.
Import Data
Meteobridge PRO and NANO SD can inport data from various sources, including
- Meteohub monthly raw data
- Meteoplug monthly esport data
- Meteotemplate
- Cumulus
- weewx
- WSWIN
Import operations can take some time as thousands of data sets need to be stored in Meteobridge's internal database. In order to import a file it needs to be dropped into the "import" folder of Meteobridge's PC-network share. Import can be told to stop on errors or to ignore those. Import operation is logged in messages section of "System" tab.
Export Data
Meteobridge PRO's or NANO SD's PC network folder contains a "templates" sub folder where export templates are stored. Template drop-down menu provides predefined standard templates for data export but will also present user-defined export templates (files with extension ".exp").
Export Template Format
A template file consist of lines beginning with a "#", which are copied 1:1 (without the leading "#") to the output files, and lines that are handled as a variable templates where variable definitions are replaced with sensor data from the archive. The example below shows an example that exports
- a date and time stamp
- outdoor temperature in °F
- outdoor humidity in %
- dew point in °F
- sea level pressure in inHg
## Standard CSV template with data in imperial units ## ## date, time, temperature[F], humidity[%], dew point[F], sealevel pressure[inHg] [YYYY]-[MM]-[DD],[hh]:[mm],[th0temp-avg=F.1:],[th0hum-avg.0:],[th0dew-avg=F.1:],[thb0seapress-avg=inhg.1:]
Meteobridge PRO also defines pragmas, that control how special data situations are handled. Pragmas are lines that start with "#pragma" and contain some of the following control settings:
- emptylines=n blocks lines that don't have any sensor data from being printed into the target file. Default is that empty lines are printed (emptylines=y).
- futuredata=n blocks lines that have time stamps in the future. Default is futuredata=n.
- crlf=y provides Windows style line ends (CR + LF) instead of Linux style (LF only). Default is crlf=n.
- log=y logs data export execution in Meteobridsge system messages. Default is log=y.
- fillgap=120 gives a time period data of a sensor is assumed to be recent. During that period the data of sensors is repeated into following lines of export data in case the sensor has not provided fresh data. Default is 600 seccond (10 minutes).
- exportfile=myexport.txt specifies a dedicated name of the export file. By default the name of the template and the chosen time period is used to generate a file name automatically.
Starting Exports
Depending on the drill-down in the "Inspect Stored Data" section "Export Data" field presents export options ranging from exporting data of a day to data of a whole year. This exports can be done in resolution of a minute, hour, day or month. Combination of export of a whole year's data and resolution down to a minute is not allowed as this will produce too much data. All other combinations can be done.
Exported data is stored on the PC network folder in subfolder "export" and is named accordingly to the used template, the time range and the used time resolution. To give an example, "standard-iso-201511-day.txt" is an export generated by export template "standard-iso", it is data from November 2015 and data is provided in a day resolution. Data in the file might look like this:
# Standard CSV template with data in metric units # # date, time, temperature[C], humidity[%], dew point[C], sealevel pressure[hPa] 2015-11-01,00:00,,,, 2015-11-02,00:00,,,, 2015-11-03,00:00,,,, 2015-11-04,00:00,,,, 2015-11-05,00:00,,,, 2015-11-06,00:00,,,, 2015-11-07,00:00,,,, 2015-11-08,00:00,,,, 2015-11-09,00:00,,,, 2015-11-10,00:00,,,, 2015-11-11,00:00,13.8,87,11.6,1018.8 2015-11-12,00:00,13.0,85,10.5,1021.3 2015-11-13,00:00,,79,7.3,1015.6 2015-11-14,00:00,7.9,86,5.7,1015.7 2015-11-15,00:00,9.9,89,8.1,1004.4 2015-11-16,00:00,11.9,88,10.0,1007.9 2015-11-17,00:00,11.4,89,9.5,1002.7 2015-11-18,00:00,11.9,87,9.8,1002.2 2015-11-19,00:00,10.4,88,8.5,1002.4 2015-11-20,00:00,8.6,89,6.8,999.3 2015-11-21,00:00,6.3,88,4.5,994.1 2015-11-22,00:00,2.6,86,0.5,1001.2 2015-11-23,00:00,3.8,87,1.9,1019.1 2015-11-24,00:00,4.5,86,2.3,1012.9 2015-11-25,00:00,5.3,89,3.7,1003.1 2015-11-26,00:00,6.0,88,4.1,1013.7 2015-11-27,00:00,5.4,86,3.4,1013.7 2015-11-28,00:00,6.3,86,4.2,1004.1 2015-11-29,00:00,,,, 2015-11-30,00:00,,,,
</translate>
