en:programming_tasks
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
en:programming_tasks [2013/08/06 09:29] – kaklik | en:programming_tasks [Unknown date] (current) – external edit (Unknown date) 127.0.0.1 | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | ====== | + | ====== |
+ | ===== Distributed network measurement system ===== | ||
- | ===== Ideas ===== | + | The network is intened for collection and distribution of measurements recorded on amateur volunteer stations. The resulting data are publicly available for analysis. |
- | - Extend open-source toolkit which can be used as an substitute for closed source programs. | + | {{ : |
- | - Design and implement reliable algorithms for distinguishing different types of meteors. | + | |
- | ==== Open-source meteor detection toolkit | + | ==== |
- | === Brief explanation | + | |
- | [[en: | + | * Meteorological data from stations |
+ | * Radio meteor detection stations [[cs: | ||
+ | * Video meteor | ||
+ | * Data reported by accidental observers - [[http:// | ||
+ | * Seismic / infrasonic measurements (meteor impacts, explosions, ...) | ||
+ | * Radio telescopes | ||
+ | * Parallel observations from several astronomy sites | ||
+ | * Geomagnetic measurements | ||
+ | * SID monitors | ||
+ | * FRB monitors | ||
+ | * GRB monitors | ||
+ | * Network for lightning detection | ||
+ | * Measurement and coordination network nodes [[cs: | ||
+ | * Nodes acting as data storage, or performing some calculations on the data (data mining servers) | ||
- | == Software for RMDS01A == | + | Each node as part of a network will be operated by the observational application (eg, [[en: |
- | [[en:RMDS]] | + | ==== System Properties |
- | Most important pieces | + | - Measuring network should be able to send short informational messages with low data volume as alert of special events. For example, in the case of Fireball Network a bolide detection which is expected |
- | For data acquisition | + | - The station should be configurable through |
- | For data synchronization with other nodes a set of [[http://www.mlab.cz/WebSVN/listing.php? | + | - Central data data should have an API that would allow programming of other applications that use measured data (for live outputs in observatories and planetariums, |
+ | - An important parameter | ||
- | Some screenshots from [[https:// | ||
- | {{: | ||
- | {{: | ||
- | ==Proposed projects:== | + | === Typical station design === |
- | - **Difficulty**: | + | |
- | - **Difficulty**: | + | |
- | - **Difficulty**: | + | |
- | - **Difficulty**: | + | |
- | // | + | {{ : |
+ | Image above illustrates an typical station connected to the distributed measurement system. The measuring station itself consist several subsystems. | ||
- | === Expected results === | + | == Local presentation node == |
- | Single package | + | A PC system located at same site where the measuring station operates but generally in another room or building. |
- | === Knowledge prerequisites === | + | == Measuring node == |
- | * Knowledge | + | This is part which performs the measurement and it is responsible to data making. Measuring node has several minor subsystems. |
- | | + | |
- | * Ability | + | Measuring Hardware - A set of sensors and hardware required for measuring. |
- | | + | |
- | | + | Hardware station guard - A hardware recovery unit which could separately reset every hardware part of measuring node. The station-supervisor daemon should use this device for error outputs. Connection management between recovery unit and station-supervisor daemon will be self controlled by device. Interface between hardware recovery unit and main station computer should be terminal based. |
+ | standard system console) in system console mode station | ||
+ | supervisor-should write debug output on this device (similarly | ||
+ | for example) | ||
+ | |||
+ | == Network interface == | ||
+ | |||
+ | We will not have direct IP connectivity from Internet to stations. Station supervisor software should generate data file outputs and upload these data files to an web server which serves station status information. Telemetry information will be displayed in the web on separate central web server. | ||
+ | |||
+ | Moreover web access is intended as file based. File based means that | ||
+ | station-supervisor uploads an diagnostic file to the central web server. Web server takes this files (from more than one station) and serve a web page with stations status information. User interface in this case is limited only to modification of an station-supervisor configuration file stored | ||
+ | on web server. | ||
+ | |||
+ | For example configuration file may contain an parameter for station reboot | ||
+ | request. After station-supervisor will notice a change or update in the | ||
+ | web server stored configuration file, station-supervisor reboots the station and | ||
+ | update its parameters according to actual configuration file. | ||
+ | |||
+ | |||
+ | === Node software === | ||
+ | |||
+ | Node hardware will run a data-processing utility ([[en: | ||
+ | |||
+ | Station-supervisor does not have a screen output. Screen output may be served by client program ssmon (Station Supervisor Monitor) which generates interactive status screen after connection to Station Supervisor server. | ||
+ | |||
+ | |||
+ | ==== Hardware implementation ==== | ||
+ | |||
+ | === Computing power === | ||
+ | |||
+ | Measuring stations should be equipped with basic computing hardware architecture based on [[cs: | ||
+ | |||
+ | |||
+ | === Network connectivity === | ||
+ | |||
+ | Parameters of network connectivity depends on the specific measurements ongoing at the station. The preferred option but will connection to ethernet. Alternatively wifi connection to the local network and then to the Internet. In some cases, remote stations and measurement that are not required large data flows can also consider using [[cs: | ||
+ | |||
+ | === Time synchronization === | ||
+ | |||
+ | Accuracy requirements for the station time again depends on the type of measurement. In the case of stations connected to the network via Ethernet, it is possible | ||
+ | |||
+ | ==== Software implementation | ||
+ | |||
+ | The network management would be advantageous | ||
+ | |||
+ | === Data Processing | ||
+ | |||
+ | Using [[http:// | ||
+ | |||
+ | === Generating secondary outputs === | ||
+ | |||
+ | The measured data of the stations should be distributed to networks specialized for processing certain types of data. (RMOB for example). List of currently available specialized networks follows. | ||
+ | |||
+ | == Meteorological data == | ||
+ | |||
+ | In the case of meteorological data are available, for example the following networks: | ||
+ | |||
+ | * [[http:// | ||
+ | * [[http:// | ||
+ | |||
+ | |||
+ | == Radio meteor observations == | ||
+ | |||
+ | Data should be processed into colorgramm graph and forwarded | ||
+ | |||
+ | === System stations administration | ||
+ | |||
+ | Registration of individual stations | ||
+ | |||
+ | Each station would then authenticate into the system with its private key. Which would be generated either by the server based on the user registration to the system, or directly to a user who would put his public key available to the network. | ||
+ | |||
+ | === Measurement data storage and archive | ||
+ | |||
+ | The measured data are primarily backed by the station (on the data node). Each station will send data in to decentralized repository which includes a central cache server | ||
+ | |||
+ | {{ : | ||
+ | |||
+ | The first step towards decentralized data storage will be the implementation of a centralized server which will receive station data directly (FTP, SSH / Rsync). With the gradual development of software tools enabling decentralization of data, data nodes will be added to which the station will be able to write. | ||
+ | |||
+ | Later the management of large volume of data will be handled exclusively in a distributed storage that will act also as part of the network. Data would therefore have redundancy based on division among multiple users, whose improve available data capacity and also data throughput. The data could then be exchanged between nodes by P2P technology, or other existing distributed data storage system. | ||
+ | |||
+ | Decentralized organization such storage | ||
+ | |||
+ | |||
+ | ==== Data Visualization ==== | ||
+ | |||
+ | === Web Interface === | ||
+ | |||
+ | Head echo meteor | ||
+ | |||
+ | Marking would be carried out by volunteers processors. Similarly, as in the project [[https:// | ||
+ | |||
+ | |||
+ | == Radio records replaying == | ||
+ | |||
+ | Live waterfall and audio output generated in HTML5 webpage. (Similar solution is [[http:// | ||
+ | |||
+ | == Display of meteorological data == | ||
+ | |||
+ | Record of the wind direction should be combined with the speed and displayed in 3D, as the deformation of the cylindrical surfaces. | ||
+ | |||
+ | A similar solution | ||
+ | |||
+ | === FITS rendering === | ||
+ | |||
+ | Ideally, Fits viewer | ||
+ | |||
+ | Existing Web browser FITS: [[https:// | ||
+ | |||
+ | The second possibility is to generate a preview by scripting on the server and erasing the oldest unused files (from the limited data space), | ||
+ | |||
+ | === Local Data Viewer === | ||
+ | |||
+ | It should contain a computationally intensive functions that can not be handled a web browser. | ||
+ | |||
+ | == Viewing image based recordings | ||
+ | |||
+ | Visualization system for multi-station experiments, | ||
+ | |||
+ | Further improvement could be to use OpenGL for rendering textures of large images. | ||
+ | |||
+ | == Current FITS viewers == | ||
+ | |||
+ | * ds9 | ||
+ | * qfitsview | ||
+ | * fits liberator | ||
+ | * http:// | ||
+ | |||
+ | == Multi parametric | ||
+ | |||
+ | A client application to download | ||
+ | |||
+ | ==== Known projects with a similar goal ==== | ||
+ | |||
+ | * [[http:// | ||
+ | * [[http:// | ||
+ | * [[https:// | ||
+ | |||
+ | ===== Integrated SDR receiver | ||
+ | |||
+ | The compact MLAB SDR receiver integrated in the box [[cs: | ||
+ | |||
+ | * [[http:// | ||
+ | |||
+ | |||
+ | ===== Astrozor ===== | ||
+ | |||
+ | Adding support for users identification on observatories: | ||
+ | |||
+ | -The possibility of using an identification card | ||
+ | -Identify the user to the observatory on the basis of information about mobile phones IMEI, MAC IP, Bluetooth MAC (paired with bluetooth). | ||
+ | -A device mounted on observing site which would be capable to retrieve identification information. | ||
+ | |||
+ | Other [[http:// | ||
+ | |||
+ | ===== Meteograms for astronomers | ||
+ | |||
+ | Generating view of clouds in the sky from a particular observation point. Meteogram in the form of video could also display the time evolution of cloud. | ||
+ | |||
+ | For some cases, | ||
+ | |||
+ | The result of this calculation could be used for observation planning. | ||
+ | |||
+ | ==== References ==== | ||
+ | |||
+ | * [[https:// | ||
+ | * [[http:// | ||
+ | ===== Implementation resources ===== | ||
+ | |||
+ | ==== Investigators ==== | ||
+ | |||
+ | The project is implemented by a team of several students CTU, TU and members of Robozor robotics club: | ||
+ | |||
+ | * Martin Povišer - Signal digitalization and storage decentralization | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | |||
+ | ==== Founding ==== | ||
+ | |||
+ | The project was supported by the ESA program of ESA Summer of Code 2013 and 2014. | ||
+ | Most hardware part of the project is realized from support of [[http:// | ||
- | ===== Test ===== | ||
- | We have several [[en: | ||
- | ===== Contact ===== | ||
- | For futher information we strongly recomends to cantact project coordinator Jakub Kakona (kaklik@mlab.cz) | ||
- | Pokud navíc umíte český jazyk, tak je vhodné si též prohlédnout [[cs: |
en/programming_tasks.1375781395.txt.gz · Last modified: 2013/08/06 09:29 (external edit) · Currently locked by: 65.108.99.55