Open-Source Tool for Enhancing Climate Resillience - DZone Open Source
source link: https://dzone.com/articles/open-source-schema-driven-asset-management
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
This article is an introduction to a community-led open-source critical asset management project called CAMS. The article provides some context to the project origin as well as some examples of the application schema and queries to provide an understanding of its build. It is based on a graph database and is aimed at helping nations, cities, and communities build their climate resilience.
Hurricane Maria
Climate change is warming the temperatures of the seas. This is causing dire troubles for island nations, cities, and communities. Tropical storms are becoming more frequent and ferocious and battering these places with merciless force.
The Commonwealth of Dominica is one example of having to suffer the consequences of climate change. In September 2017, they were battered by Hurricane Maria, a category five hurricane. With winds of 160 miles per hour, it destroyed 90 percent of the island’s structures and crops, caused $1.3 billion in loss (equivalent to 224% of Dominica’s GDP), and resulted in the loss of 65 lives.
Dominica understood that Maria was not a one-off event and has made the pledge to become the world’s first climate-resilient nation.
An Open-Source Project Led by the United Nations
Dominica’s Department of Planning, Department of Industry and Commerce, and the special-created Climate Execution Agency for Dominica began work to build their resilience to climate change.
The United Nations Office for Disaster Risk Reduction leaned on the support of their private-sector advisory group, ARISE-US. ARISE-US is part of the ARISE Global Network, a team of private-sector volunteers helping nations big and small prevent disasters so their enterprises and communities can thrive.
The Problem To Solve
ARISE-US discovered vulnerabilities in Dominica’s knowledge about their critical asset infrastructure.
The island’s critical assets such as hospitals, electricity, water, sanitation, transportation, and communications are run by a multitude of governmental and private entities. Information about these assets is also stored across different formats, such as spreadsheets and word documents, and there is no central repository for first responders and disaster planners to work from.
As important as understanding and identifying its critical assets, Dominica also needed an understanding of the relationships between assets. For example, if one fails, what does it impact, and if the impacted asset fails what does that affect? This is the cascading failure chain.
First responders and disaster planning agencies needed an application where they can plot their assets on a map and visually see their relationships, and also have the ability to factor in the type of event (hurricane, flooding, etc.) to enable Dominica to plan for, and respond to extreme weather events.
Using a Graph Database to Prioritize Relationships
ARISE-US built a team of technical volunteers to help them develop an application to help Dominica’s goal, with skills spanning earth sciences, spatial management, data analytics, and graph databases.
The critical asset management system, known as CAMS, is built upon TerminusDB, which is a document graph database. The TerminusDB schema language enables documents and their relationships to be specified using simple JSON syntax. This syntax lets users specify a JSON object to automatically convert to a graph.
Due to the need for relationships to be at the forefront of the application, graph database technology was the logical choice.
The team spent time working with Dominica to understand their problems and started the project by building the schema, which you can see here:
[
{
"@base": "iri://CAMS/",
"@schema": "iri://CAMS#",
"@type": "@context"
},
{
"@id": "LineString_Type",
"@type": "Enum",
"@value": [
"LineString"
]
},
{
"@documentation": {
"@comment": "Update history",
"@properties": {
"comment": "A comment relating to an historic hazard incident.",
"date": "The date at which the update occurred."
}
},
"@id": "UpdateEvent",
"@inherits": "Event",
"@key": {
"@fields": [
"comment",
"date"
],
"@type": "Lexical"
},
"@subdocument": [],
"@type": "Class",
"comment": "xsd:string"
},
{
"@id": "GradedHazard",
"@key": {
"@type": "Random"
},
"@subdocument": [],
"@type": "Class",
"Grade": {
"@class": "xsd:decimal",
"@type": "Optional"
},
"hazard": {
"@class": "Hazard",
"@type": "Optional"
}
},
{
"@abstract": [],
"@id": "Event",
"@subdocument": [],
"@type": "Class",
"date": "xsd:dateTime"
},
{
"@id": "HazardScale",
"@key": {
"@type": "Random"
},
"@type": "Class",
"hazard": "Hazard",
"max": "xsd:decimal",
"min": "xsd:decimal"
},
{
"@abstract": [],
"@id": "AssetType",
"@type": "Class",
"name": "xsd:string"
},
{
"@id": "Owner",
"@type": "Class",
"contact_person": "Person",
"name": "xsd:string"
},
{
"@id": "SpatialWebIdentifier",
"@type": "Class",
"id": "xsd:string"
},
{
"@id": "CRS84_Type",
"@type": "Enum",
"@value": [
"urn:ogc:def:crs:OGC:1.3:CRS84"
]
},
{
"@id": "CRS84",
"@inherits": "Properties",
"@type": "Class",
"name": "CRS84_Type"
},
{
"@abstract": [],
"@id": "Properties",
"@type": "Class"
},
{
"@id": "DependencyRelation",
"@type": "Class",
"comment": "xsd:string",
"critical": "xsd:boolean",
"dependent": "Asset",
"depends_on": "Asset"
},
{
"@id": "Feature",
"@type": "Class",
"bbox": {
"@class": "xsd:string",
"@dimensions": 1,
"@type": "Array"
},
"centerline": {
"@class": "Geometry",
"@type": "Optional"
},
"geometry": "Geometry",
"id": {
"@class": "xsd:string",
"@type": "Optional"
},
"properties": "Properties",
"title": {
"@class": "xsd:string",
"@type": "Optional"
},
"type": "Feature_Type"
},
{
"@id": "Person",
"@type": "Class",
"email_address": {
"@class": "xsd:string",
"@type": "Optional"
},
"first_name": "xsd:string",
"job_title": {
"@class": "xsd:string",
"@type": "Optional"
},
"last_name": "xsd:string",
"organization": {
"@class": "xsd:string",
"@type": "Optional"
},
"phone_number": {
"@class": "xsd:string",
"@type": "Optional"
}
},
{
"@id": "Name_Type",
"@type": "Enum",
"@value": [
"name"
]
},
{
"@id": "OSiProperties",
"@inherits": "Properties",
"@type": "Class",
"NAMN1": "xsd:string",
"OBJECTID": "xsd:integer"
},
{
"@abstract": [],
"@id": "Geometry",
"@key": {
"@type": "Random"
},
"@subdocument": [],
"@type": "Class"
},
{
"@id": "Area",
"@type": "Class",
"extent": {
"@class": "AreaExtent",
"@type": "Optional"
},
"hazard_history": {
"@class": "HazardEvent",
"@type": "Set"
},
"hazards": {
"@class": "Hazard",
"@type": "Set"
},
"name": "xsd:string",
"population": {
"@class": "xsd:integer",
"@type": "Optional"
}
},
{
"@abstract": [],
"@id": "FundingSource",
"@type": "Class"
},
{
"@id": "MultiPolygon_Type",
"@type": "Enum",
"@value": [
"MultiPolygon"
]
},
{
"@abstract": [],
"@id": "Source",
"@type": "Class"
},
{
"@id": "GeoCoordinate",
"@key": {
"@fields": [
"latitude",
"longitude"
],
"@type": "Lexical"
},
"@subdocument": [],
"@type": "Class",
"latitude": "xsd:decimal",
"longitude": "xsd:decimal"
},
{
"@id": "Polygon_Type",
"@type": "Enum",
"@value": [
"Polygon"
]
},
{
"@id": "name",
"@type": "Class",
"properties": "Properties",
"type": "Name_Type"
},
{
"@id": "Point_Type",
"@type": "Enum",
"@value": [
"Point"
]
},
{
"@id": "FeatureCollection_Type",
"@type": "Enum",
"@value": [
"FeatureCollection"
]
},
{
"@id": "Hazard",
"@type": "Enum",
"@value": [
"Volcanos (incl. lahars, pyroclastic flows, volcanic activity)",
"Landslides (incl. post wildfire landslides) and Avalanches",
"Hurricanes, Typhoons, or Cyclones",
"Tropical/Extra Tropical of other extreme storms",
"Coast Storm Surge",
"Pluvial and Fluvial Flooding",
"\"Sunny Day\" Tidal Flooding",
"Tornadoes, Derechos, Micro-Bursts",
"Lightning Strikes",
"Wildfires",
"Drought",
"Geologic Sink Holes",
"Pest Infestations",
"Famine",
"High Temperature Event",
"Low Temperature Event",
"Cyber Attack or Failure",
"Other Terrorism",
"Industrial Accident (Emissions, Releases, Spills, Ect.)",
"Earthquakes"
]
},
{
"@id": "FeatureCollection",
"@type": "Class",
"crs": {
"@class": "name",
"@type": "Optional"
},
"features": {
"@class": "Feature",
"@type": "Set"
},
"name": {
"@class": "xsd:string",
"@type": "Optional"
},
"type": "FeatureCollection_Type"
},
{
"@id": "AreaExtent",
"@key": {
"@type": "ValueHash"
},
"@subdocument": [],
"@type": "Class",
"perimeter": {
"@class": "xsd:integer",
"@type": "Optional"
}
},
{
"@documentation": {
"@comment": "Historical hazard",
"@properties": {
"comment": "A comment relating to an historic hazard incident.",
"date": "The date at which the incident occurred."
}
},
"@id": "HazardEvent",
"@inherits": "Event",
"@key": {
"@fields": [
"hazard",
"date"
],
"@type": "Lexical"
},
"@subdocument": [],
"@type": "Class",
"comment": "xsd:string",
"hazard": "Hazard"
},
{
"@id": "Asset",
"@key": {
"@fields": [
"asset_identifier"
],
"@type": "Lexical"
},
"@type": "Class",
"applicable_hazards": {
"@class": "GradedHazard",
"@type": "Set"
},
"assetType": {
"@class": "AssetEnum",
"@type": "Optional"
},
"asset_history": {
"@class": "Event",
"@type": "Set"
},
"asset_identifier": "xsd:string",
"asset_update_history": {
"@class": "UpdateEvent",
"@type": "Set"
},
"commisioning_date": "xsd:dateTime",
"description": {
"@class": "xsd:string",
"@type": "Set"
},
"design_standards": "xsd:string",
"last_maintained": "xsd:dateTime",
"last_modified": "xsd:dateTime",
"location": "Location",
"name": "xsd:string",
"owner": {
"@class": "Owner",
"@type": "Set"
},
"spatial_web_identifier": {
"@class": "SpatialWebIdentifier",
"@type": "Optional"
}
},
{
"@id": "Point",
"@inherits": "Geometry",
"@key": {
"@type": "Random"
},
"@subdocument": [],
"@type": "Class",
"coordinates": {
"@class": "xsd:decimal",
"@dimensions": 1,
"@type": "Array"
},
"type": "Point_Type"
},
{
"@id": "Feature_Type",
"@type": "Enum",
"@value": [
"Feature"
]
},
{
"@id": "GeometryCollection_Type",
"@type": "Enum",
"@value": [
"GeometryCollection"
]
},
{
"@id": "Location",
"@key": {
"@type": "Random"
},
"@subdocument": [],
"@type": "Class",
"city": "xsd:string",
"geometry_location": {
"@class": "Geometry",
"@type": "Optional"
},
"postal_code": {
"@class": "xsd:string",
"@type": "Optional"
},
"state": "xsd:string",
"street": "xsd:string"
},
{
"@id": "AssetEnum",
"@type": "Enum",
"@value": [
"Police Station",
"Fire Stations",
"Hospital/Medical Clinic",
"Government Buildings",
"Shelters/ Special Needs",
"Marine Ports",
"Airport",
"Electrical Power Generating Plants",
"Water System",
"Desalinization Plant",
"Desalination Plant",
"Water Distribution System"
]
},
{
"@id": "GeoPerimeter",
"@key": {
"@fields": [
"perimeter"
],
"@type": "Lexical"
},
"@subdocument": [],
"@type": "Class",
"perimeter": {
"@class": "GeoCoordinate",
"@type": "List"
}
}
]
Here is the graph view.
CAMS is an open-source project and the schema and instructions to set up your own instance of the application are all included within the GitHub repository.
To help you visualize the types of relationships the application is mapping for Dominica see the illustration below:
Critical assets have dependency relationships between them, but they also feature other relationships.
- Are they impacted by specific events?
- Are they impacted by a specific severity of an event?
- Are they in a location that is susceptible to particular events, such as a flood plain?
As these relationships are edges in the graph, the application can use the edges within queries to provide the functionality required to visually display dependencies on a map using geocoordinates.
Query and UI
The database schema includes the graph's edges and provides a foundation for UI development. A feature of TerminusDB is that document frames are automatically generated from the schema and the UI has been built directly from these frames using the data management platform’s SDK. All development has been achieved using the React JS Library.
Using a datalog query language, the relationships between assets and events have been embedded into the application. Some of the queries are listed below to give you a look under the hood:
//query to get all assets
export const getAvailableAssets = () => {
let WOQL= TerminusDBClient.WOQL
return WOQL.triple("v:Asset", "rdf:type","@schema:Asset")
.triple("v:Asset", "@schema:name", "v:Name")
.opt(WOQL.triple("v:Asset", "@schema:description", "v:Description"))
.triple("v:Asset", "@schema:asset_identifier", "v:AssetIdentifier")
.opt(WOQL.triple("v:Asset", "@schema:assetType", "v:AssetType"))
.triple("v:Asset", "@schema:design_standards", "v:DesignStandards")
.triple("v:Asset", "@schema:last_maintained", "v:LastMaintained")
.triple("v:Asset", "@schema:location", "v:Location")
.triple("v:Location", "@schema:geometry_location", "v:Point")
.triple("v:Point", "@schema:coordinates", "v:AssetCoordinatesX")
.triple("v:Point", "@schema:coordinates", "v:AssetCoordinatesY")
.triple("v:AssetCoordinatesX", "sys:value", "v:AssetX")
.triple("v:AssetCoordinatesX", "sys:index", "v:X_AssetX")
.eq("v:X_AssetX", 0)
.triple("v:AssetCoordinatesY", "sys:value", "v:AssetY")
.triple("v:AssetCoordinatesY", "sys:index", WOQL.literal(1, "xsd:nonNegativeInteger"))
}This query displays all of the assets on a map and is used when a user first logs into CAMS. It also displays relevant information to the user so they can quickly access important data about an asset.
// query to show dependency relations between assets
export const getAssetDependentOnQuery = (documentID) =>{
let WOQL= TerminusDBClient.WOQL
return WOQL.triple("v:DependencyRelation", "@schema:depends_on", documentID)
.triple("v:DependencyRelation", "@schema:critical", "v:Critical")
.triple("v:DependencyRelation", "@schema:dependent", "v:Asset")
.triple("v:Asset", "@schema:name", "v:Name")
.opt(WOQL.triple("v:Asset", "@schema:description", "v:Description"))
.triple("v:Asset", "@schema:asset_identifier", "v:AssetIdentifier")
.opt(WOQL.triple("v:Asset", "@schema:assetType", "v:AssetType"))
.triple("v:Asset", "@schema:design_standards", "v:DesignStandards")
.triple("v:Asset", "@schema:last_maintained", "v:LastMaintained")
.triple("v:Asset", "@schema:location", "v:DependentLocation")
.triple("v:DependentLocation", "@schema:geometry_location", "v:Point")
.triple("v:Point", "@schema:coordinates", "v:AssetCoordinatesX")
.triple("v:Point", "@schema:coordinates", "v:AssetCoordinatesY")
.triple("v:AssetCoordinatesX", "sys:value", "v:AssetX")
.triple("v:AssetCoordinatesX", "sys:index", "v:X_AssetX")
.eq("v:X_AssetX", 0)
.triple("v:AssetCoordinatesY", "sys:value", "v:AssetY")
.triple("v:AssetCoordinatesY", "sys:index", WOQL.literal(1, "xsd:nonNegativeInteger"))
}When a user selects an asset on the map within the UI, based on the documentID, the application will then query the schema to return the dependency relationships with other assets. These are direct dependencies to that particular documentID. When combined with the query below, the application then displays the failure chains to other linked critical assets.
// failure chain query - Linked Asset is dependants
export const getAssetFailureChain = (asset) => {
let WOQL=TerminusDBClient.WOQL
let documentID=asset
//let documentID = "Asset/Castle%20bruce%20Electrical%20substation%20"
return WOQL.and(
WOQL.path(documentID, "(<depends_on,dependent>)*", "v:Asset"),
WOQL.triple("v:Inter", "@schema:depends_on", "v:Asset"),
WOQL.triple("v:Inter", "@schema:dependent", "v:LinkedAsset"),
)
.triple("v:LinkedAsset", "@schema:name", "v:LinkedAssetName")
.opt(WOQL.triple("v:LinkedAsset", "@schema:description", "v:LinkedAssetDescription"))
.opt(WOQL.triple("v:LinkedAsset", "@schema:assetType", "v:LinkedAssetType"))
.triple("v:LinkedAsset", "@schema:location", "v:LinkedAssetLocation")
.triple("v:LinkedAssetLocation", "@schema:geometry_location", "v:LinkedAssetPoint")
.triple("v:LinkedAssetPoint", "@schema:coordinates", "v:LinkedAssetCoordinatesX")
.triple("v:LinkedAssetPoint", "@schema:coordinates", "v:LinkedAssetCoordinatesY")
.triple("v:LinkedAssetCoordinatesX", "sys:value", "v:LinkedAssetX")
.triple("v:LinkedAssetCoordinatesX", "sys:index", "v:X_LinkedAssetX")
.eq("v:X_LinkedAssetX", 0)
.triple("v:LinkedAssetCoordinatesY", "sys:value", "v:LinkedAssetY")
.triple("v:LinkedAssetCoordinatesY", "sys:index", WOQL.literal(1, "xsd:nonNegativeInteger"))
.triple("v:Asset", "@schema:location", "v:AssetLocation")
.triple("v:AssetLocation", "@schema:geometry_location", "v:AssetPoint")
.triple("v:AssetPoint", "@schema:coordinates", "v:AssetCoordinatesX")
.triple("v:AssetPoint", "@schema:coordinates", "v:AssetCoordinatesY")
.triple("v:AssetCoordinatesX", "sys:value", "v:AssetX")
.triple("v:AssetCoordinatesX", "sys:index", "v:X_AssetX")
.eq("v:X_AssetX", 0)
.triple("v:AssetCoordinatesY", "sys:value", "v:AssetY")
.triple("v:AssetCoordinatesY", "sys:index", WOQL.literal(1, "xsd:nonNegativeInteger"))
}As one final query example, it’s important for users to be able to specify an event and severity to show susceptible assets. For example, if a hurricane is forecast, being able to pinpoint where disaster planners need to focus will help plan and mitigate disasters. This query combines document properties with edges to only show assets impacted by a particular event:
// query to get assets filtered by hazard events
export const getAssetsByEventsOrIDQuery = (event, asset) => {
let WOQL= TerminusDBClient.WOQL
let documentID=false, eventQuery=false
if(asset) documentID=asset
//console.log("event", event)
if(event && Object.keys(event).length && event.hasOwnProperty("eventName")) {
let eventName = encodeURI(event.eventName.trim())
let hazard = `@schema:Hazard/${eventName}`
eventQuery = WOQL.triple("v:Asset", "@schema:applicable_hazards", "v:Hazard")
.triple("v:Hazard", "@schema:hazard", hazard)
.opt(WOQL.triple("v:Hazard", "@schema:Grade", "v:Grade"))
if(event.hasOwnProperty("grade") && event.grade){ // get grade match
eventQuery.not(WOQL.greater("v:Grade", Number(event.grade)))
//.less("v:Grade", Number(event.grade))
//eventQuery.triple("v:Hazard", "@schema:Grade", Number(event.grade))
}
}MVP and Beyond
The CAMS team has built the MVP for Dominica over the past couple of months, but there is still much to be done to provide a richer user experience and help Dominica, and other cities, island nations, and communities use it to build their resilience to climate change.
Dominica is using CAMS to plan for hurricane season to build their climate resilience and is being provided free of charge, all work being done voluntarily.
Open Source for Good
The CAMS project is open source and is being provided as a free service to those who need to build their climate resilience. The MVP has been launched, but there is a lot of work to do to provide even more functionality, such as graph analytics to help nations analyze historical responses, automated alerts, and mobile applications that function offline when power and connectivity cannot be guaranteed.
If you’re interested in getting involved in an open-source product for good, check out the CAMS website and GitHub repo for more information.
Recommend
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK