The master's API¶

The master's operations as JSON, for a caller in another process or another language: a test rig, a script, or the console, which uses nothing else.

py1815-master console      # the API over HTTP, and the console, on 127.0.0.1:8815
py1815-master serve        # the API a line of JSON at a time, on 127.0.0.1:8816

Both take the same outstation options as the console, and both can load every setting from a JSON file with --config. See Configuring the master.

Over HTTP¶

Each operation is a route. Its parameters are a JSON object in the body, and its answer is a JSON object.

curl -s http://127.0.0.1:8815/api/add -H 'Content-Type: application/json' \
     -d '{"name": "lab", "host": "192.0.2.10", "port": 20000}'

curl -s http://127.0.0.1:8815/api/scan -H 'Content-Type: application/json' \
     -d '{"outstation": "lab", "kind": "integrity"}'

{"id": null, "ok": true, "result": {
  "function": "READ", "task": null, "sequence": 2, "outcome": "complete", "fragments": 1,
  "indications": ["NEED_TIME", "DEVICE_RESTART"], "object_count": 344,
  "elapsed_ms": 15.0, "request": "c0013c02063c03063c04063c0106", "undecoded": [],
  "objects": [{"type": "bi", "group": 1, "variation": 2, "index": 0, "name": null,
               "value": false, "flags": ["ONLINE"], "time_ms": null,
               "synchronized": null, "event": false}]}}

The whole API is described in an OpenAPI document, which the service serves at /openapi.json and the repository keeps beside the console. It is built from one description of the operations, and a test fails when the service and the document disagree.

Route	Does
`POST /api/status`	The outstations, and how each stands
`POST /api/add`	Adds an outstation and connects to it
`POST /api/remove`	Closes an outstation's connection and forgets it
`POST /api/connect`, `POST /api/disconnect`	Its connection. `connect` keeps trying for `wait` seconds when given
`POST /api/idle`	Waits until nothing the master does unasked is due or under way
`POST /api/set_tasks`	Changes what the master does for an outstation unasked
`POST /api/wait_for`	Waits until a value, an event or an indication is as named
`POST /api/profile`	Every point of the profile an outstation is meant to serve
`POST /api/scan`	A poll by `kind`: `integrity`, `events`, `class0` to `class3`, `outputs`
`POST /api/read`	Named points, in one request
`POST /api/values`	What has been read, with no traffic
`POST /api/events`	The events received
`POST /api/request`	Any request, by function code
`POST /api/operate`	Outputs commanded, by type and index
`POST /api/write_time`, `POST /api/clear_restart`	The clock, and the restart indication
`POST /api/synchronize_time`	The clock, by the LAN or the non-LAN procedure of IEEE 1815-2012
`POST /api/freeze`	Counters frozen
`POST /api/restart`	A cold or a warm restart
`POST /api/broadcast`	Any request, sent to a broadcast address
`POST /api/enable_unsolicited`, `POST /api/disable_unsolicited`	Reporting by event class
`POST /api/repeat`	A scan on a schedule, or an end to one
`POST /api/trace`	The frames sent and received
`POST /api/capture`	The frames kept, as a pcap file in base64
`POST /api/clear`	Forgets the frames or the events kept
`POST /api/deviate`	Turns named deviations on or off: protocol rules the master breaks on purpose
`POST /api/checks`	The checks an evaluation can run
`POST /api/evaluate`	Runs checks against an outstation, and answers with the report
`POST /api/der.read`	Points of the DER profile by name, or a named group, in engineering units
`POST /api/der.write`	Outputs of the DER profile by name, in engineering units
`POST /api/der.enable`, `POST /api/der.disable`	A DER function, and then whether it is enabled
`POST /api/der.functions`	The DER functions, which are supported and which enabled
`POST /api/der.curve`	The curve the curve block shows
`POST /api/der.write_curve`	A curve: its selector, its fields and its points, then read back
`POST /api/der.compare`	What the outstation serves, against its Device Profile document
`POST /api/der.document`	A Device Profile document laid out for reading: header facts, and sections of tables
`POST /api/stop`	Ends the service
`POST /api`	Any of the above as one message: see below
`GET /events`	What happens unasked, as server-sent events
`GET /openapi.json`	The description of these routes

A request body is sent as application/json, always. Anything else is refused, which is part of how the service tells its own page from someone else's.

What an answer means¶

{"id": null, "ok": true, "result": {}}
{"id": null, "ok": false, "error": {"kind": "request", "message": "there is no outstation named 'lab'"}}

ok says whether the operation was carried out. It is false in three cases:

`error.kind`	Means
`request`	The message could not be acted on: an operation that does not exist, an outstation nobody added, a parameter of the wrong kind
`connection`	There was no connection to ask over, or one could not be made
`not_allowed`	The operation commands the outstation, and the service was not started to. See Commanding

An outstation that did not answer is neither. The operation was carried out, ok is true, and the result's outcome is timeout. The same goes for a request the outstation refused: outcome is complete, and indications names what it refused with.

`outcome`	Means
`complete`	The response arrived, to its final fragment
`timeout`	Nothing more arrived in time. What did arrive is in the result
`sent`	The request takes no response, or went to a broadcast address, and was sent
`abandoned`	The request was given up on while it was outstanding: the connection ended, or the scan on a schedule that made it was set again

What the master does by itself¶

An outstation added through the service is looked after as the master looks after any: it is settled when the connection is made, its events are fetched when it says it has some, and a connection that is lost is made again. add says otherwise:

curl -s http://127.0.0.1:8815/api/add -H 'Content-Type: application/json' \
     -d '{"name": "lab", "host": "192.0.2.10",
          "tasks": {"enable_unsolicited": [1, 2, 3], "events_when_indicated": false},
          "reconnect": 1.0}'

Parameter	Is
`tasks`	An object of choices by task: `startup`, `clear_restart`, `write_time`, `events_when_indicated` and `integrity_on_overflow`, each true or false, and `enable_unsolicited`, a list of event classes. A task left out stands at its default
`manual`	True to send nothing that was not asked for: no task, and no confirmation. `tasks` and `confirm` given beside it are kept as given
`reconnect`	Seconds between attempts to make a lost connection again. Five when left out, and null for never
`read_retries`	Times a read that times out with nothing received is sent again, under the same sequence number. None when left out. No other request is ever sent again
`tls`	Connect over TLS: an object of `ca`, `certificate`, `key` and `server_name`, files on the machine the service runs on. See Configuring the master

Each exchange says, in retries, how many times its request was sent again, and in broadcast, the broadcast address it went to or null.

set_tasks changes the tasks of an outstation already added. What it gives is merged with what the outstation has, and a task that was due and is turned off is not done:

curl -s http://127.0.0.1:8815/api/set_tasks -H 'Content-Type: application/json' \
     -d '{"outstation": "lab", "tasks": {"events_when_indicated": false}}'

add and connect answer as soon as the connection is made, and idle answers once what is done on connecting has been done. An outstation's entry in status gives its tasks, the tasks_due that are waiting, and reconnect. A request the master made for a task has the task's name in task, in the exchange event that reports it; a request that was asked for has null.

Two of the tasks write to the outstation: clear_restart and write_time. In a service that was not started to command they are off, and asking for either by name is refused as not_allowed. Such a service settles an outstation and leaves its restart indication and its clock as it found them. Started with --allow-control, it does both. set_tasks refuses to turn either on in the same way.

connect with wait keeps trying, once a second, for that many seconds when the connection cannot be made, and answers with the last connection error when the time is up. Without it the first failure is the answer.

Commanding¶

A service reads unless it is started to command:

py1815-master console --allow-control
py1815-master serve --allow-control

Without that, every operation that changes an outstation is refused with not_allowed and nothing is sent: operate, write_time, synchronize_time, clear_restart, freeze, restart, der.write, der.enable, der.disable, der.write_curve, der.curve given a curve to select, and a request or a broadcast by any function code other than READ, ENABLE_UNSOLICITED, DISABLE_UNSOLICITED and DELAY_MEASURE. The two tasks that write are off as well. status says which it is, in allow_control.

curl -s http://127.0.0.1:8815/api/operate -H 'Content-Type: application/json' \
     -d '{"outstation": "lab", "points": {"bo": {"17": true}, "ao": {"88": 500}}, "mode": "select"}'

{"id": null, "ok": true, "result": {
  "mode": "select", "operated": true, "accepted": true,
  "statuses": [
    {"type": "bo", "index": 17, "name": null, "variation": 1, "value": 3,
     "status": "SUCCESS", "echoed": true},
    {"type": "ao", "index": 88, "name": null, "variation": 2, "value": 500,
     "status": "SUCCESS", "echoed": true}],
  "exchanges": [{"function": "SELECT", "outcome": "complete"},
                {"function": "OPERATE", "outcome": "complete"}]}}

Field	Holds
`points`	The outputs: `bo` and `ao`, each an object of values by index. A binary output takes `true` or `false` for a latch, or an operation by name; an analog output takes a number
`mode`	`direct`, `select` for a select and then an operate, or `direct_no_ack`
`operated`	Whether a request that operates was sent. False after a select the outstation refused
`accepted`	True when every control was accepted, false when one was refused, null when the outstation never said
`statuses`	For each control, the status the outstation answered with, by name, and whether it echoed the control as sent
`exchanges`	Each request made, as any other exchange is described

A refused control is an answer, with ok true: the operation was carried out and the outstation said no. accepted is null when the request takes no acknowledgment or the response did not arrive. The service never sends a control twice.

A whole number is sent as an integer and anything else as a float, unless variation says which. An outstation that scales its points takes an integer as the transmitted value and a float as the engineering one.

The clock, by procedure¶

curl -s http://127.0.0.1:8815/api/synchronize_time -H 'Content-Type: application/json' \
     -d '{"outstation": "lab", "procedure": "non_lan"}'

procedure is lan, the default, or non_lan. The answer has written, whether the write was sent, which it is only after the first request was answered without an error; accepted, whether the outstation took it; delay_ms, the one-way delay non_lan measured; time_ms, the time written; and exchanges, the two requests. Neither is ever sent again.

Broadcast¶

curl -s http://127.0.0.1:8815/api/broadcast -H 'Content-Type: application/json' \
     -d '{"outstation": "lab", "function": "IMMED_FREEZE_NR", "body": "140006",
          "address": "shall_confirm"}'

The request goes to address: no_confirm (0xFFFD), shall_confirm (0xFFFE) or optional_confirm (0xFFFF, the default), which say whether the outstation asks for the response that reports the broadcast to be confirmed. No outstation answers, so the exchange's outcome is sent. It travels on the outstation's connection, and the outstation reports it with BROADCAST in its next response.

The DER profile¶

An outstation given a profile, with --profile or profile in its configuration, is an IEEE 1815.2 DER, and the der.* operations speak to it in the profile's terms. See Commanding for the ones that write: each is a control, sent once, and refused with not_allowed by a service that does not command.

curl -s http://127.0.0.1:8815/api/der.read -H 'Content-Type: application/json'      -d '{"outstation": "lab", "names": ["AI148", "System Meter Active Power"]}'

curl -s http://127.0.0.1:8815/api/der.write -H 'Content-Type: application/json'      -d '{"outstation": "lab", "points": {"AO87": 50}, "verify": true}'

{"id": null, "ok": true, "result": {
  "accepted": true, "verified": true,
  "points": [{"address": "AO87", "type": "ao", "index": 87, "units": "Percent of WMax",
              "name": "Active Power Limit Maximum (Pos = Generation, Neg = Consumption)",
              "requested": 50.0, "sent": 500, "sent_value": 50.0,
              "status": "SUCCESS", "echoed": true, "matches": true,
              "readback": {"address": "AI148", "value": 50.0, "raw": 500,
                           "quality": "good", "flags": ["ONLINE"]}}],
  "operated": {"mode": "direct", "operated": true, "accepted": true}}}

A point is named by its address, as in AO87, or by its name in the tables, whole or its first sentence, ignoring case. The tables give some names to more than one point: a read looks among the inputs first, a write among the outputs only, and a name that still means two points is refused with both addresses.

Operation	Takes	Does
`der.read`	`names`, `group`, or both	Reads the points in one request. A `group` is a function, by its key, name, purpose or enable output, or everything of one purpose, as in `Nameplate` or `Monitoring`. Each point comes back with its `value` in engineering units, its `raw` value, its `state` by name for a binary point, its `flags` and its `quality`
`der.write`	`points`: values by name or address; `verify`, `mode`, `variation`	Writes the outputs in one request, in the order given. An analog value is divided by the point's multiplier and sent as the nearest whole number, and one outside the point's range is refused before anything is sent. A binary output takes `true`, `false`, or a state by name. With `verify` the input that mirrors each output is read afterwards, and `matches` says whether it is within one step of the multiplier
`der.enable`, `der.disable`	`function`, `mode`	Latches the function's enable output on or off, then reads the input that reports whether it is enabled: `enabled`
`der.functions`		Reads every function's supports input and enabled input in one request, and lists each function with its settings, their units and ranges
`der.curve`	`number`, `mode`	Reads the curve the curve block shows. Given `number`, writes the selector first, which commands
`der.write_curve`	`number`, `type`, `x_units`, `y_units`, `points`, `mode`	Writes the selector, then the type, the number of points and the units, then X and Y of each point, each only when the one before it was accepted, and reads the curve back. `stopped_at` names the write that was not accepted, and `matches` says whether the curve read back is the one written
`der.compare`	`document`	Compares what the outstation serves with a DNP3 Device Profile document: the one sent, or the one given with `--device-profile`
`der.document`	`document`	Lays a DNP3 Device Profile document out for reading: the one sent, or the one given with `--device-profile`. Sends nothing to the outstation, and needs no `outstation` when `document` is sent

A read that the outstation refuses whole, because one range in it holds no point it serves, is made again a range at a time, and a point no read returned is not_reported. A disabled function's inputs come back offline: IEEE 1815.2 has them sent without ONLINE.

A curve's points are the numbers that travel. The tables give the curve block no multiplier, because the units the curve declares say how its points scale, and type_name, x_units_name and y_units_name give the names the tables list for each.

der.compare reads class 0, output status, and then every declared point neither returned. absent lists what is declared and was not returned, undeclared what was returned and is not declared, and class_0 the served points a class 0 read does or does not carry against the declaration. A point's event class and deadband cannot be asked of an outstation, so points lists them as declared. It needs no profile.

A Device Profile document, laid out¶

der.document reads the XML of a Device Profile document and answers with it as a person reads it. It takes a document of any edition of the schema, from any vendor, since it reads element names and not a schema:

curl -s -X POST http://127.0.0.1:8815/api/der.document \
     -H 'Content-Type: application/json' -d '{"outstation": "lab"}'

{"ok": true, "result": {
  "title": "EX-100 Device Profile",
  "notice": "Laid out by py1815 from the Device Profile XML. This is py1815's own layout, not the DNP Users Group's.",
  "facts": [{"label": "Schema version", "value": "2.12.00"},
            {"label": "Vendor name", "value": "Example Co"}],
  "sections": [
    {"title": "Configuration", "tables": [
      {"title": "Link config",
       "columns": ["Item", "Capabilities", "Current value"],
       "rows": [["Data link address", "0 to 65519", "1024"],
                ["Source address validation", "Never\nAlways single address", "Always single address"]]}]},
    {"title": "Data points list", "tables": [
      {"title": "Analog input points",
       "columns": ["Index", "Name", "Change event class", "Units"],
       "rows": [["4", "Active power", "two", "W"]]}]}]}}

Field	Holds
`title`	The document's name, or the device's when the document has none
`notice`	One line saying whose layout this is, to show with the document
`facts`	What the document header states, and the device's vendor, name and versions
`sections`	The parts of the document in its own order: the header's revision history, the configuration, the database, the implementation table and the point lists, and any part an edition or a vendor adds

A table of settings has a row for each setting, with the columns Item, Capabilities and Current value, Methods when a setting says how it is configured, and Stated for anything else a setting holds. The implementation table and each point list have a row for each entry and a column for each property the entries carry. Every cell is text. A cell that states several things separates them with a newline, and a cell the document says nothing for is empty.

The layout is this library's own. A label is the element's own name made readable, as in Max received fragment size for maxReceivedFragmentSize, so it is not the wording of the DNP Users Group's stylesheet, and an element the layout does not recognize is still shown, by its label and its text.

Nothing is asked of the outstation, so the operation is answered by a service that was not started to command, for an outstation that is not connected, and with no outstation at all when document is sent. A document that is not XML, is not a Device Profile, declares a document type, or is larger than 8 MiB is refused with a request error that says which, and so is one whose tables would be wider or larger than the reader lays out. Over HTTP a message of more than 4 MiB is refused before that, with status 413, and over the line socket a line of more than 4 MiB is answered with a request error and the connection is closed.

Values¶

An object out of a response, in objects and in events:

Field	Holds
`type`	`bi`, `bo`, `counter`, `frozen`, `ai` or `ao`, or null for an object that is not a point's value
`group`, `variation`, `index`	Where the object sits in DNP3's terms
`name`	The point's name, when the outstation has a profile
`value`	A boolean, a count, or an analog value as transmitted
`flags`	The flag bits set, by name, or null for a variation that carries none
`time_ms`	The outstation's time for the value, in milliseconds since the epoch, when it gave one
`event`	Whether it reports a change, and not the present value

A point of a profile, from profile:

Field	Holds
`index`, `name`	The point, named as the profile's tables name it
`label`	The name without the values it lists, for a point that is an enumeration
`enumeration`	Those values, each a `value` and a `name`, or null
`mandatory`	Whether the profile requires every outstation to implement it
`section`	The heading the tables list it under

Waiting for something¶

wait_for answers when a condition holds, or at timeout seconds, and says which in held. Nothing is sent to make it hold. It takes exactly one of:

Condition	Holds once
`value`	The point `type` and `index` has been reported, with the value named when one is: `equals`, within `tolerance` of it, `at_least` or `at_most`
`event`	An event arrives after the wait began, of the `type` and `index` given when they are
`indication`	The indication `name` is set in the last response, or clear when `set` is false

curl -s http://127.0.0.1:8815/api/wait_for -H 'Content-Type: application/json' \
     -d '{"outstation": "lab", "timeout": 10,
          "value": {"type": "ao", "index": 87, "equals": 5000}}'

The answer has held, elapsed_ms, the last indications, the point a value condition names as it stands, and the events that satisfied an event condition.

A capture¶

capture answers with the frames kept for an outstation as a pcap file, in base64 so that JSON can carry it, and the number of frames in it. Decoded, it opens in Wireshark:

curl -s http://127.0.0.1:8815/api/capture -H 'Content-Type: application/json' \
     -d '{"outstation": "lab"}' | jq -r .result.pcap | base64 -d > lab.pcap

after keeps only the frames with a greater id, as it does for trace. The file's layout is described under Captures. It reads nothing from the outstation, so a service that does not command answers it.

What happens unasked¶

GET /events is a stream of server-sent events that stays open. Each event's data is a JSON object:

`event`	Sent when	Carries
`frame`	A frame is sent or received	`outstation`, `frame`
`exchange`	A request ends: asked for, repeated, or made for a task	`outstation`, `exchange`
`unsolicited`	An unsolicited response arrives	`outstation`, `indications`, `objects`
`connection`	A connection is made or ends	`outstation`, `connected`
`check`	A check of an evaluation ends	`outstation`, `result`
`outstations`	An outstation is added or removed
`lost`	This subscriber fell 10,000 updates behind, and the waiting updates were dropped	`dropped`: how many

A subscriber that receives lost should read the state it shows again, for example with status, values, events and trace. It stays subscribed. An event stream that has not accepted a write for 60 seconds is closed; open a new one.

Every operation that commands an outstation is written to the master's log at INFO with the points and values asked for and the result, and one refused for lack of --allow-control at WARNING.

curl -N http://127.0.0.1:8815/events

An evaluation¶

evaluate runs the checks of Evaluating an outstation against an outstation that was given a profile, and answers with the report when the last check has ended. checks lists them.

curl -s http://127.0.0.1:8815/api/evaluate -H 'Content-Type: application/json' \
    -d '{"outstation": "lab", "checks": ["MON-001", "deviation"], "allow_deviations": true}'

Parameter	Meaning
`checks`	Check identifiers or sets. All of them when left out
`allow_deviations`	Run the checks that have the master break a protocol rule. False when left out
`settle`	Seconds to wait for the outstation to reach a state it was commanded to. 5 when left out
`curves`	How many curves the outstation stores. Found by selecting each when left out

The answer is the report py1815-master evaluate --report writes. A run takes as long as its checks do, which is minutes for all of them, so each result is also sent as a check event as its check ends. A check that writes is run only when the service was started with --allow-control, and is then written to the master's log as a command. A check that may not be run is reported as not_run. One evaluation runs against an outstation at a time, and a second is refused while it does. While a check that deviates is under way, every other request to that outstation waits for the check to end.

One message at a time¶

POST /api takes any operation as one message: the operation by name, its parameters, and an id the answer repeats.

{"id": 7, "op": "scan", "outstation": "lab", "params": {"kind": "integrity"}}

py1815-master serve takes the same messages, one to a line over a local TCP socket, and answers each with one line. A test rig starts it, adds its outstations, and ends it with stop:

import json
import socket

rig = socket.create_connection(("127.0.0.1", 8816)).makefile("rw")


def ask(op, outstation=None, **params):
    rig.write(json.dumps({"op": op, "outstation": outstation, "params": params}) + "\n")
    rig.flush()
    return json.loads(rig.readline())


ask("add", name="lab", host="192.0.2.10", port=20000)
poll = ask("scan", "lab", kind="integrity")["result"]
print(poll["outcome"], poll["object_count"])

Send {"op": "subscribe"} and the same connection is also sent the events above, one to a line. The line service listens on this machine only.

A token¶

Started with a token, the service requires it of every request to /api and /events, as Authorization: Bearer <token>, or as a token query parameter where a header cannot be set, as on the event stream. The console's own files, and /openapi.json, need none. See Who can reach it.