Master¶

A master for exercising outstations. The master, The console, The master's API and Evaluating an outstation are the guides; this is the reference.

Over a socket¶

py1815.master.api ¶

A master on a socket: the interface a script uses.

:class:Master holds outstations by name. :class:Outstation is one association over one TCP connection, with TLS when it is given a context, and is where the requests are. Each request is a coroutine that returns an :class:~py1815.master.association.Exchange saying what happened: a timeout and an error indication are results, and an exception means the interface was misused or the connection could not be made.

Each outstation also runs automatic tasks: a startup sequence on connect, clearing the restart indication, writing the time on request, polling for events when indicated, and reconnecting after a lost connection. Configure them with :class:~py1815.master.tasks.Tasks, or pass manual=True to disable every task and confirmation. :mod:py1815.master.sync has the same interface for a script with no event loop.

RetryReport ¶

Log failed reconnection attempts without filling the log.

The first failure is logged at INFO. Later failures are logged at DEBUG, with one INFO record each interval seconds that counts them. An outstation that is down for a day with a five-second retry then writes about 25 INFO records instead of 17,000.

Source code in src/py1815/master/api.py

class RetryReport:
    """Log failed reconnection attempts without filling the log.

    The first failure is logged at INFO. Later failures are logged at DEBUG,
    with one INFO record each ``interval`` seconds that counts them. An
    outstation that is down for a day with a five-second retry then writes
    about 25 INFO records instead of 17,000.
    """

    def __init__(
        self,
        name: str,
        *,
        interval: float = RETRY_REPORT_INTERVAL,
        clock: Callable[[], float] = time.monotonic,
    ) -> None:
        self.name = name
        self.attempts = 0
        self._interval = interval
        self._clock = clock
        self._started = clock()
        self._reported: float | None = None

    def failed(self, error: OSError) -> int:
        """Record one failed attempt, and return the level it was logged at."""
        self.attempts += 1
        now = self._clock()
        if self._reported is None or now - self._reported >= self._interval:
            self._reported = now
            level = logging.INFO
            minutes = (now - self._started) / 60
            logger.log(
                level,
                "dnp3 master: %s is not reachable (%d failed attempt(s) in %.0f min): %s",
                self.name,
                self.attempts,
                minutes,
                error,
            )
        else:
            level = logging.DEBUG
            logger.log(level, "dnp3 master: %s is still not reachable: %s", self.name, error)
        return level

failed ¶

failed(error: OSError) -> int

Record one failed attempt, and return the level it was logged at.

Source code in src/py1815/master/api.py

def failed(self, error: OSError) -> int:
    """Record one failed attempt, and return the level it was logged at."""
    self.attempts += 1
    now = self._clock()
    if self._reported is None or now - self._reported >= self._interval:
        self._reported = now
        level = logging.INFO
        minutes = (now - self._started) / 60
        logger.log(
            level,
            "dnp3 master: %s is not reachable (%d failed attempt(s) in %.0f min): %s",
            self.name,
            self.attempts,
            minutes,
            error,
        )
    else:
        level = logging.DEBUG
        logger.log(level, "dnp3 master: %s is still not reachable: %s", self.name, error)
    return level

NotConnected ¶

Bases: ConnectionError

A request was made of an outstation that has no connection.

Source code in src/py1815/master/api.py

class NotConnected(ConnectionError):
    """A request was made of an outstation that has no connection."""

Outstation ¶

Bases: Operations[Awaitable[Exchange], Awaitable[Operated], Awaitable[Synchronized]]

One outstation, as a master sees it: a connection, an association, a store.

Source code in src/py1815/master/api.py

class Outstation(Operations[Awaitable[Exchange], Awaitable[Operated], Awaitable[Synchronized]]):
    """One outstation, as a master sees it: a connection, an association, a store."""

    def __init__(
        self,
        name: str,
        *,
        host: str,
        port: int = 20000,
        outstation_address: int = 1024,
        master_address: int = 1,
        response_timeout: float = DEFAULT_RESPONSE_TIMEOUT,
        read_retries: int = DEFAULT_READ_RETRIES,
        connect_timeout: float = DEFAULT_CONNECT_TIMEOUT,
        tls: ssl.SSLContext | None = None,
        server_name: str | None = None,
        confirm: bool | None = None,
        tasks: Tasks | None = None,
        manual: bool = False,
        reconnect: float | None = DEFAULT_RECONNECT,
        on_unsolicited: Callable[[Unsolicited], None] | None = None,
        on_exchange: Callable[[Exchange], None] | None = None,
        on_connection: Callable[[bool], None] | None = None,
        connection_observer: ConnectionObserver | None = None,
    ) -> None:
        """
        Args:
            name: What the caller calls this outstation.
            host: Where it listens.
            port: The port it listens on.
            outstation_address: Its link address.
            master_address: The link address this master speaks from.
            response_timeout: Seconds to wait for a response, and for each
                further fragment of one.
            read_retries: Times a read that times out with nothing received
                is sent again, under the same sequence number. No other
                request is sent again. See
                :class:`~py1815.master.association.MasterAssociation`.
            connect_timeout: Seconds to wait for the connection to be made,
                and for the TLS handshake when there is one.
            tls: Connect over TLS with this client context. An outstation
                that requires TLS also requires a certificate of the master:
                load one into the context. :mod:`py1815.master.tls` makes one
                from files.
            server_name: The name the outstation's certificate is checked
                against, when it is not ``host``. Only with ``tls``.
            confirm: Whether fragments that ask to be confirmed are. See
                :class:`~py1815.master.association.MasterAssociation`.
                Defaults to True, or False when ``manual`` is set.
            tasks: The automatic tasks to run. Defaults to ``Tasks()``, or
                ``Tasks.none()`` when ``manual`` is set.
            manual: Disable all automatic tasks and confirmations, so only
                requests the caller makes are sent. An explicit ``tasks`` or
                ``confirm`` overrides this.
            reconnect: Seconds between reconnection attempts after a
                connection is lost, or None to disable reconnection. A
                connection ended by :meth:`close` is not reopened.
            on_unsolicited: Called with each unsolicited response as it
                arrives, after it has been confirmed and stored.
            on_exchange: Called with each exchange as it ends, after its
                objects have been stored.
            on_connection: Called with True when the connection is made and
                False when it ends.
            connection_observer: Called with a
                :class:`~py1815.observers.ConnectionEvent` for each attempt
                to connect, made or failed, each connection lost, and each
                connection closed by :meth:`close`, with the endpoints and
                the reason. Settable later through the attribute of the same
                name. See :mod:`py1815.observers`.
        """
        self.name = name
        self.host = host
        self.port = port
        if reconnect is not None and not (reconnect > 0 and math.isfinite(reconnect)):
            raise ValueError(f"reconnect is {reconnect}; it is a wait between attempts")
        if server_name is not None and tls is None:
            raise ValueError("server_name names the certificate of a TLS connection; give tls")
        self.tls = tls
        self.server_name = server_name
        self.association = MasterAssociation(
            outstation_address=outstation_address,
            master_address=master_address,
            response_timeout=response_timeout,
            confirm=not manual if confirm is None else confirm,
            read_retries=read_retries,
        )
        if tasks is None:
            tasks = Tasks.none() if manual else Tasks()
        self._housekeeper = Housekeeper(tasks)
        self.reconnect = reconnect
        self.store = Store()
        self.trace = Trace()
        self.on_unsolicited = on_unsolicited
        self.on_exchange = on_exchange
        self.on_connection = on_connection
        #: Called with each connection event, or None.
        self.connection_observer = connection_observer
        self._connection_guard = ObserverGuard("connection observer")
        #: The report of the connection made, kept to report its end.
        self._connected_event: ConnectionEvent | None = None
        #: The protocol rules the master is breaking on purpose, off by default.
        #: Applied to what the association produced, on its way to the channel.
        self.deviations = Deviations()
        #: How the exchanges so far ended, by outcome.
        self.counts: dict[str, int] = {}
        #: The last exchange that received anything, and when it ended.
        self.last_response: Exchange | None = None
        self.last_response_at: float | None = None
        #: The indications of the last response of either kind, solicited or
        #: unsolicited, or None before the first.
        self.indications: IIN | None = None
        self._scanning: dict[str, asyncio.Task[None]] = {}
        self._scan_intervals: dict[str, float] = {}
        self._connect_timeout = connect_timeout
        self._unsolicited: deque[Unsolicited] = deque(maxlen=_UNSOLICITED_KEPT)
        self._reader: asyncio.StreamReader | None = None
        self._writer: asyncio.StreamWriter | None = None
        self._receiving: asyncio.Task[None] | None = None
        self._housekeeping: asyncio.Task[None] | None = None
        self._reconnecting: asyncio.Task[None] | None = None
        #: Set whenever something arrived or the connection ended, so a
        #: waiting request looks again.
        self._progress = asyncio.Event()
        #: Set when a task becomes due, to wake the task worker.
        self._attention = asyncio.Event()
        #: Set and cleared at once whenever the store, the indications or the
        #: connection may have changed, to wake whoever is in wait_for().
        self._changed = asyncio.Event()
        #: Set while no task is pending or running.
        self._settled = asyncio.Event()
        self._settled.set()
        #: One request at a time, in the order asked.
        self._turn = asyncio.Lock()
        #: The task holding the outstation across several requests, if one is.
        self._reserved_by: asyncio.Task[Any] | None = None

    # ------------------------------------------------------------ connection

    @property
    def connected(self) -> bool:
        return self._writer is not None and not self._writer.is_closing()

    @property
    def tasks(self) -> Tasks:
        """The automatic task settings for this outstation.

        Set it to change them while the outstation is connected. A task that
        is due and is now off is not done; what is done next follows the new
        settings.
        """
        return self._housekeeper.tasks

    @tasks.setter
    def tasks(self, tasks: Tasks) -> None:
        if not isinstance(tasks, Tasks):
            raise TypeError(f"tasks is a Tasks, not {type(tasks).__name__}")
        self._housekeeper.tasks = tasks
        if not self._housekeeper.due:
            self._settled.set()

    @property
    def tasks_due(self) -> tuple[str, ...]:
        """The pending tasks, in execution order."""
        return self._housekeeper.due

    @property
    def unsolicited(self) -> tuple[Unsolicited, ...]:
        """The unsolicited responses received, oldest first."""
        return tuple(self._unsolicited)

    async def connect(
        self, *, wait: float | None = None, every: float = DEFAULT_CONNECT_EVERY
    ) -> None:
        """Open the connection. Raises ``OSError`` if it cannot be made.

        With ``wait``, a connection that cannot be made is tried again every
        ``every`` seconds for up to ``wait`` seconds, and the last error is
        raised when none could be made in that time. Without it the first
        failure is raised.

        Returns as soon as the connection is open. The startup tasks run
        after that; use :meth:`idle` to wait for them.

        If this is called while the master is retrying a lost connection and
        the attempt fails, the error is raised and the retries carry on.
        """
        if wait is not None and not (wait >= 0 and math.isfinite(wait)):
            raise ValueError(f"wait is {wait}; it is seconds, zero or more")
        if not (every > 0 and math.isfinite(every)):
            raise ValueError(f"every is {every}; it is a wait between attempts")
        loop = asyncio.get_running_loop()
        deadline = loop.time() + (wait or 0.0)
        retrying = self._reconnecting is not None
        await self._stop_reconnecting()
        while True:
            try:
                await self._open()
                return
            except OSError as error:
                if loop.time() + every > deadline:
                    # This call cancelled the retry worker. Start it again,
                    # so a failed manual attempt does not end automatic
                    # reconnection.
                    if retrying:
                        self._start_reconnecting()
                    raise
                logger.info("dnp3 master: %s is not reachable yet: %s", self.name, error)
            await asyncio.sleep(every)

    async def idle(self) -> None:
        """Wait until no automatic task is pending or running.

        Returns immediately if there is none, or if the connection has ended.
        """
        await self._settled.wait()

    async def _open(self) -> None:
        if self.connected:
            return
        secure: dict[str, Any] = {}
        if self.tls is not None:
            secure["ssl"] = self.tls
            if self.server_name is not None:
                secure["server_hostname"] = self.server_name
        try:
            reader, writer = await asyncio.wait_for(
                asyncio.open_connection(self.host, self.port, **secure), self._connect_timeout
            )
        except TimeoutError as exc:
            self._report_failure("timeout")
            raise TimeoutError(
                f"no connection to {self.host}:{self.port} within {self._connect_timeout} s"
            ) from exc
        except OSError as exc:
            self._report_failure(_failure_reason(exc))
            raise
        self._reader, self._writer = reader, writer
        self.association.connection_reset()
        self.association.take()
        self.trace.reset(
            local=_address(writer.get_extra_info("sockname")),
            peer=_address(writer.get_extra_info("peername")),
        )
        self._receiving = asyncio.create_task(self._receive(), name=f"dnp3-master-{self.name}")
        logger.info("dnp3 master: connected to %s at %s:%d", self.name, self.host, self.port)
        self._report_connected(writer)
        self._notify_connection(True)
        # Queue the startup tasks for the new connection.
        self._housekeeper.connected()
        self._attention.clear()
        if self._housekeeper.due:
            self._settled.clear()
        self._housekeeping = asyncio.create_task(
            self._keep_house(), name=f"dnp3-master-{self.name}-tasks"
        )
        # Yield once so the task worker acquires the request lock first.
        # Requests made after this returns, including repeated scans, then
        # queue behind the startup tasks.
        await asyncio.sleep(0)
        for kind, interval in self._scan_intervals.items():
            self._start_scan(kind, interval)

    async def _stop_reconnecting(self) -> None:
        reconnecting, self._reconnecting = self._reconnecting, None
        if reconnecting is not None:
            # Wait for the cancellation, so an attempt in progress cannot
            # complete afterwards and open a second connection.
            reconnecting.cancel()
            with contextlib.suppress(asyncio.CancelledError):
                await reconnecting

    def _stop_housekeeping(self) -> None:
        housekeeping, self._housekeeping = self._housekeeping, None
        if housekeeping is not None:
            housekeeping.cancel()
        # No tasks run without a connection, so release anyone in idle().
        # Set it here as well as in the worker: a worker cancelled before its
        # first run never reaches its own cleanup.
        self._settled.set()

    def _start_reconnecting(self) -> None:
        """Start the retry worker, unless reconnection is off or one is running."""
        if self.reconnect is not None and self._reconnecting is None:
            self._reconnecting = asyncio.create_task(
                self._reconnect(self.reconnect), name=f"dnp3-master-{self.name}-reconnect"
            )

    async def _reconnect(self, interval: float) -> None:
        report = RetryReport(self.name)
        while True:
            await asyncio.sleep(interval)
            try:
                await self._open()
            except OSError as exc:
                report.failed(exc)
                continue
            # The peer can close the new connection before _open() returns.
            # The receive loop does not start another worker while this one
            # is running, so keep retrying here.
            if self.connected:
                logger.info(
                    "dnp3 master: connected to %s again after %d failed attempt(s)",
                    self.name,
                    report.attempts,
                )
                self._reconnecting = None
                return
            logger.info("dnp3 master: %s closed the new connection at once", self.name)
            self._report_failure("closed at once")

    async def close(self) -> None:
        """Close the connection. The store and the association's counters stay."""
        await self._stop_reconnecting()
        self._stop_housekeeping()
        for task in self._scanning.values():
            task.cancel()
        self._scanning.clear()
        receiving, self._receiving = self._receiving, None
        writer, self._writer = self._writer, None
        self._reader = None
        if receiving is not None:
            receiving.cancel()
            with contextlib.suppress(asyncio.CancelledError):
                await receiving
        if writer is not None:
            writer.close()
            with contextlib.suppress(OSError):
                await writer.wait_closed()
            self._report_end(ConnectionOutcome.ENDED, "closed by the master")
        self.association.connection_reset()
        self._progress.set()

    async def _receive(self) -> None:
        reader, writer = self._reader, self._writer
        assert reader is not None and writer is not None
        reason = "error"
        try:
            while True:
                data = await reader.read(_READ_SIZE)
                if not data:
                    logger.info("dnp3 master: %s closed the connection", self.name)
                    reason = "closed by peer"
                    return
                self.trace.record(RECEIVED, data)
                reply = self.association.receive(data)
                if reply:
                    await self._send(writer, reply, kind=deviations.CONFIRM)
                self._deliver_unsolicited()
                self._progress.set()
        except OSError as exc:
            logger.info("dnp3 master: connection to %s lost: %s", self.name, exc)
            reason = _failure_reason(exc, "error")
        finally:
            # Whatever was outstanding went with the connection. Say so, and
            # wake whoever was waiting for it.
            self.association.connection_reset()
            if self._writer is writer:
                self._writer = None
                writer.close()
                self._stop_housekeeping()
                self._report_end(ConnectionOutcome.LOST, reason)
                self._notify_connection(False)
                # The connection was lost, not closed by the caller: reconnect.
                self._start_reconnecting()
            self._progress.set()

    def _signal_change(self) -> None:
        """Wake every wait_for(), each to look at its condition again.

        The event is set and replaced, not set and cleared: a waiter holds the
        event it took before it looked at its condition, so a change between
        the look and the wait still wakes it.
        """
        self._changed.set()
        self._changed = asyncio.Event()

    def _notify_connection(self, connected: bool) -> None:
        self._signal_change()
        if self.on_connection is not None:
            try:
                self.on_connection(connected)
            except Exception:  # pylint: disable=broad-exception-caught
                logger.exception("dnp3 master: the connection handler for %s failed", self.name)

    def _report(self, event: ConnectionEvent) -> None:
        """Pass a connection event to the observer, if there is one."""
        self._connection_guard.deliver(self.connection_observer, event)

    def _event(self, outcome: ConnectionOutcome, **fields: Any) -> ConnectionEvent:
        """Return a connection event of this outstation's connection."""
        return ConnectionEvent(
            role=ConnectionRole.MASTER,
            direction=ConnectionDirection.OUTBOUND,
            outcome=outcome,
            transport=ConnectionTransport.TCP if self.tls is None else ConnectionTransport.TLS,
            remote=Endpoint(self.host, self.port),
            name=self.name,
            **fields,
        )

    def _report_failure(self, reason: str) -> None:
        """Report an attempt to connect that failed."""
        self._report(self._event(ConnectionOutcome.FAILED, reason=reason))

    def _report_connected(self, writer: asyncio.StreamWriter) -> None:
        """Report the connection made, and keep the report for its end."""
        event = self._event(
            ConnectionOutcome.CONNECTED,
            local=Endpoint.of(writer.get_extra_info("sockname")),
            peer_identity=self._peer_identity(writer),
        )
        remote = Endpoint.of(writer.get_extra_info("peername"))
        if remote is not None:
            event = replace(event, remote=remote)
        self._connected_event = event
        self._report(event)

    def _report_end(self, outcome: ConnectionOutcome, reason: str) -> None:
        """Report the end of the connection made, once."""
        connected, self._connected_event = self._connected_event, None
        if connected is not None:
            self._report(replace(connected, outcome=outcome, reason=reason, timestamp=time.time()))

    def _peer_identity(self, writer: asyncio.StreamWriter) -> str | None:
        """Return who the outstation proved to be over TLS, or None without TLS.

        The name its certificate was checked against when the context checks
        one, and otherwise the certificate's fingerprint.
        """
        if self.tls is None:
            return None
        if self.tls.check_hostname:
            return self.server_name or self.host
        ssl_object = writer.get_extra_info("ssl_object")
        der = ssl_object.getpeercert(binary_form=True) if ssl_object is not None else None
        return "sha256:" + hashlib.sha256(der).hexdigest() if der else None

    def _deliver_unsolicited(self) -> None:
        for unsolicited in self.association.take_unsolicited():
            self.store.apply(unsolicited.objects, now=time.monotonic())
            self._unsolicited.append(unsolicited)
            self.indications = unsolicited.iin
            self._signal_change()
            self._note(unsolicited.iin)
            if self.on_unsolicited is not None:
                try:
                    self.on_unsolicited(unsolicited)
                except Exception:  # pylint: disable=broad-exception-caught
                    # A caller's handler that raises must not end the receive
                    # loop: the connection is not at fault.
                    logger.exception(
                        "dnp3 master: the unsolicited handler for %s failed", self.name
                    )

    # -------------------------------------------------------------- requests

    @contextlib.asynccontextmanager
    async def _taking_turn(self) -> AsyncIterator[None]:
        """Take the turn to send, or go ahead when the calling task holds the outstation."""
        if self._reserved_by is not None and self._reserved_by is asyncio.current_task():
            yield
            return
        async with self._turn:
            yield

    async def _reserve(self) -> None:
        """Hold the outstation for the calling task until :meth:`_release`.

        Until then every request from another task waits its turn: the
        automatic tasks, a repeated scan, and any other caller. A check that
        has the master break a protocol rule holds the outstation, so that no
        request but its own is sent while a deviation is on, and none comes
        between its steps. Confirmations of unsolicited responses are not
        requests and are still sent.
        """
        if self._reserved_by is asyncio.current_task():
            return
        await self._turn.acquire()
        self._reserved_by = asyncio.current_task()

    def _release(self) -> None:
        """Let other tasks send again. Does nothing unless the calling task holds the outstation.

        Only the holder lets go: a task that was cancelled while it waited for
        the hold cleans up as well, and must not end another task's hold.
        """
        try:
            caller = asyncio.current_task()
        except RuntimeError:
            # Called with no event loop running, so not by the task that holds it.
            return
        if self._reserved_by is not None and self._reserved_by is caller:
            self._reserved_by = None
            self._turn.release()

    async def _exchange(
        self, function: FunctionCode, body: bytes, *, broadcast: link.Broadcast | None = None
    ) -> Exchange:
        async with self._taking_turn():
            return await self._exchange_in_turn(function, body, broadcast=broadcast)

    async def _carry_out(self, plan: Plan) -> Operated:
        # One turn for the whole plan: an operate has to follow its select
        # with no other request between them, and a scan on a schedule would
        # otherwise be free to take the gap.
        async with self._taking_turn():
            result: Operated = await self._follow(plan)
            return result

    async def _synchronize(self, plan: TimeSync) -> Synchronized:
        # One turn, so that nothing comes between the two requests and the
        # time the write carries is as fresh as it can be.
        async with self._taking_turn():
            result: Synchronized = await self._follow(plan)
            return result

    async def _follow(self, plan: Steps, *, task: str | None = None) -> Any:
        """Make the requests a plan calls for. The caller holds the turn."""
        done: list[Exchange] = []
        while (step := plan.next(done)) is not None:
            done.append(await self._exchange_in_turn(*step, task=task))
        return plan.result(done)

    async def wait_for(self, condition: Callable[[Store], bool], timeout: float) -> bool:
        """Wait until ``condition(store)`` is true, and return True; or False at the timeout.

        The condition is looked at now, and again each time a response or an
        unsolicited response has been taken into the store and
        :attr:`indications`, and when the connection is made or ends. It may
        look at anything, this outstation included:

        .. code-block:: python

            await lab.wait_for(lambda store: store.analog_output(87).value == 5000, 10)
            await lab.wait_for(lambda _: lab.indications.is_set(IINBit.NEED_TIME), 10)

        Nothing is sent to make it come true. A timeout is a result and not an
        exception; an exception the condition raises is raised here.
        """
        if not (timeout >= 0 and math.isfinite(timeout)):
            raise ValueError(f"timeout is {timeout}; it is seconds, zero or more")
        loop = asyncio.get_running_loop()
        deadline = loop.time() + timeout
        while True:
            # Taken before the look, so a change during it is not missed.
            changed = self._changed
            if condition(self.store):
                return True
            remaining = deadline - loop.time()
            if remaining <= 0:
                return False
            with contextlib.suppress(TimeoutError):
                await asyncio.wait_for(changed.wait(), remaining)

    def _note(
        self, indications: IIN | None, answering: tuple[FunctionCode, bytes] | None = None
    ) -> None:
        """Pass a response's indications to the housekeeper and wake the worker."""
        self._housekeeper.saw(indications, answering=answering)
        if self._housekeeper.due and self._housekeeping is not None:
            self._settled.clear()
            self._attention.set()

    async def _keep_house(self) -> None:
        """Task worker: send the requests for pending tasks until disconnected."""
        try:
            while True:
                if self._housekeeper.due:
                    # Hold the request lock for the whole batch, so the
                    # startup sequence is not interleaved with other requests.
                    async with self._taking_turn():
                        while (step := self._housekeeper.next()) is not None:
                            if step.plan is not None:
                                synchronized = await self._follow(step.plan, task=step.task)
                                outcome = "written" if synchronized.written else "not written"
                            else:
                                exchange = await self._exchange_in_turn(
                                    step.function, step.body, task=step.task
                                )
                                outcome = exchange.outcome.value
                            if step.task in WRITING:
                                # A write the master made by itself is a command
                                # like any other, and goes in the log as one.
                                logger.info(
                                    "dnp3 master: automatic %s to %s: %s",
                                    step.task,
                                    self.name,
                                    outcome,
                                )
                self._settled.set()
                await self._attention.wait()
                self._attention.clear()
        except NotConnected:
            logger.info("dnp3 master: the tasks for %s stopped with the connection", self.name)
        except Exception:  # pylint: disable=broad-exception-caught
            # An unexpected error is a bug in this library. Log it and stop
            # the worker without dropping the connection.
            logger.exception("dnp3 master: the tasks for %s failed", self.name)
        finally:
            self._settled.set()

    async def _exchange_in_turn(
        self,
        function: FunctionCode,
        body: bytes,
        *,
        task: str | None = None,
        broadcast: link.Broadcast | None = None,
    ) -> Exchange:
        writer = self._writer
        if writer is None or writer.is_closing():
            raise NotConnected(f"{self.name} is not connected")
        self._progress.clear()
        octets = self.association.request(function, body, broadcast=broadcast)
        try:
            await self._write(writer, octets)
            while True:
                exchange = self.association.take()
                if exchange is not None:
                    if task is not None:
                        exchange = replace(exchange, task=task)
                    self._finished(exchange, body)
                    return exchange
                wait = self.association.expires_after()
                if wait is None:
                    # Neither outstanding nor finished: the connection was
                    # closed and reopened underneath this request.
                    raise NotConnected(f"the connection to {self.name} was reset")
                if wait <= 0:
                    again = self.association.retry()
                    if again:
                        await self._write(writer, again)
                    else:
                        self.association.expire()
                    continue
                self._progress.clear()
                # Looked at again before waiting: something may have arrived
                # between taking and clearing.
                if not self.association.busy:
                    continue
                with contextlib.suppress(TimeoutError):
                    await asyncio.wait_for(self._progress.wait(), wait)
        except asyncio.CancelledError:
            # The caller cancelled while the request was in flight, for
            # example on its own timeout or when a repeated scan is
            # rescheduled. End the exchange as abandoned and record it.
            # Otherwise the association stays busy and rejects every later
            # request.
            self.association.abandon()
            given_up = self.association.take()
            if given_up is not None:
                self._finished(replace(given_up, task=task), body)
            raise

    async def _write(self, writer: asyncio.StreamWriter, octets: bytes) -> None:
        await self._send(writer, octets, kind=deviations.REQUEST)

    async def _send(self, writer: asyncio.StreamWriter, octets: bytes, *, kind: str) -> None:
        """Send octets, after any deviation has had its way with them.

        A deviation may change the octets, send them twice, or send nothing.
        The trace records each write as it was made, a frame no outstation
        accepts included, so a capture shows the deviation rather than the
        request the association built.
        """
        for write in self.deviations.outbound(octets, kind=kind):
            self.trace.record_write(write)
            writer.write(write)
        # A write that fails is seen by the receive loop as well, which ends
        # the exchange as abandoned.
        with contextlib.suppress(OSError):
            await writer.drain()

    def _finished(self, exchange: Exchange, body: bytes) -> None:
        self.store.apply(exchange.objects, now=time.monotonic())
        self._note(exchange.iin, (exchange.function, body))
        outcome = exchange.outcome.value
        self.counts[outcome] = self.counts.get(outcome, 0) + 1
        if exchange.fragments:
            self.last_response = exchange
            self.last_response_at = time.time()
            self.indications = exchange.iin
        self._signal_change()
        if self.on_exchange is not None:
            try:
                self.on_exchange(exchange)
            except Exception:  # pylint: disable=broad-exception-caught
                logger.exception("dnp3 master: the exchange handler for %s failed", self.name)

    # ----------------------------------------------------------------- scans

    @property
    def scan_intervals(self) -> dict[str, float]:
        """The scans being repeated, by kind, as seconds between them."""
        return dict(self._scan_intervals)

    def repeat_scan(self, kind: str, interval: float | None) -> None:
        """Repeat a scan every ``interval`` seconds, or stop repeating it with None.

        No scan is repeated unless this is called. A repeated scan waits its
        turn like any other request,
        stops when the connection ends, and starts again when it is made again.
        """
        requests.scan(kind)  # Refuses a kind that is not a scan, before anything runs.
        running = self._scanning.pop(kind, None)
        if running is not None:
            running.cancel()
        if interval is None:
            self._scan_intervals.pop(kind, None)
            return
        if not interval > 0:
            raise ValueError(f"an interval of {interval} is not a wait between scans")
        self._scan_intervals[kind] = interval
        if self.connected:
            self._start_scan(kind, interval)

    def _start_scan(self, kind: str, interval: float) -> None:
        running = self._scanning.pop(kind, None)
        if running is not None:
            running.cancel()
        self._scanning[kind] = asyncio.create_task(
            self._repeat(kind, interval), name=f"dnp3-master-{self.name}-{kind}"
        )

    async def _repeat(self, kind: str, interval: float) -> None:
        try:
            while True:
                await self.scan(kind)
                await asyncio.sleep(interval)
        except NotConnected:
            logger.info("dnp3 master: %s scan of %s stopped with the connection", kind, self.name)

tasks `property` `writable` ¶

tasks: Tasks

The automatic task settings for this outstation.

Set it to change them while the outstation is connected. A task that is due and is now off is not done; what is done next follows the new settings.

tasks_due `property` ¶

tasks_due: tuple[str, ...]

The pending tasks, in execution order.

unsolicited `property` ¶

unsolicited: tuple[Unsolicited, ...]

The unsolicited responses received, oldest first.

scan_intervals `property` ¶

scan_intervals: dict[str, float]

The scans being repeated, by kind, as seconds between them.

connect `async` ¶

connect(
    *,
    wait: float | None = None,
    every: float = DEFAULT_CONNECT_EVERY,
) -> None

Open the connection. Raises OSError if it cannot be made.

With wait, a connection that cannot be made is tried again every every seconds for up to wait seconds, and the last error is raised when none could be made in that time. Without it the first failure is raised.

Returns as soon as the connection is open. The startup tasks run after that; use :meth:idle to wait for them.

If this is called while the master is retrying a lost connection and the attempt fails, the error is raised and the retries carry on.

Source code in src/py1815/master/api.py

async def connect(
    self, *, wait: float | None = None, every: float = DEFAULT_CONNECT_EVERY
) -> None:
    """Open the connection. Raises ``OSError`` if it cannot be made.

    With ``wait``, a connection that cannot be made is tried again every
    ``every`` seconds for up to ``wait`` seconds, and the last error is
    raised when none could be made in that time. Without it the first
    failure is raised.

    Returns as soon as the connection is open. The startup tasks run
    after that; use :meth:`idle` to wait for them.

    If this is called while the master is retrying a lost connection and
    the attempt fails, the error is raised and the retries carry on.
    """
    if wait is not None and not (wait >= 0 and math.isfinite(wait)):
        raise ValueError(f"wait is {wait}; it is seconds, zero or more")
    if not (every > 0 and math.isfinite(every)):
        raise ValueError(f"every is {every}; it is a wait between attempts")
    loop = asyncio.get_running_loop()
    deadline = loop.time() + (wait or 0.0)
    retrying = self._reconnecting is not None
    await self._stop_reconnecting()
    while True:
        try:
            await self._open()
            return
        except OSError as error:
            if loop.time() + every > deadline:
                # This call cancelled the retry worker. Start it again,
                # so a failed manual attempt does not end automatic
                # reconnection.
                if retrying:
                    self._start_reconnecting()
                raise
            logger.info("dnp3 master: %s is not reachable yet: %s", self.name, error)
        await asyncio.sleep(every)

idle `async` ¶

idle() -> None

Wait until no automatic task is pending or running.

Returns immediately if there is none, or if the connection has ended.

Source code in src/py1815/master/api.py

async def idle(self) -> None:
    """Wait until no automatic task is pending or running.

    Returns immediately if there is none, or if the connection has ended.
    """
    await self._settled.wait()

close `async` ¶

close() -> None

Close the connection. The store and the association's counters stay.

Source code in src/py1815/master/api.py

async def close(self) -> None:
    """Close the connection. The store and the association's counters stay."""
    await self._stop_reconnecting()
    self._stop_housekeeping()
    for task in self._scanning.values():
        task.cancel()
    self._scanning.clear()
    receiving, self._receiving = self._receiving, None
    writer, self._writer = self._writer, None
    self._reader = None
    if receiving is not None:
        receiving.cancel()
        with contextlib.suppress(asyncio.CancelledError):
            await receiving
    if writer is not None:
        writer.close()
        with contextlib.suppress(OSError):
            await writer.wait_closed()
        self._report_end(ConnectionOutcome.ENDED, "closed by the master")
    self.association.connection_reset()
    self._progress.set()

wait_for `async` ¶

wait_for(
    condition: Callable[[Store], bool], timeout: float
) -> bool

Wait until condition(store) is true, and return True; or False at the timeout.

The condition is looked at now, and again each time a response or an unsolicited response has been taken into the store and :attr:indications, and when the connection is made or ends. It may look at anything, this outstation included:

.. code-block:: python

await lab.wait_for(lambda store: store.analog_output(87).value == 5000, 10)
await lab.wait_for(lambda _: lab.indications.is_set(IINBit.NEED_TIME), 10)

Nothing is sent to make it come true. A timeout is a result and not an exception; an exception the condition raises is raised here.

Source code in src/py1815/master/api.py

async def wait_for(self, condition: Callable[[Store], bool], timeout: float) -> bool:
    """Wait until ``condition(store)`` is true, and return True; or False at the timeout.

    The condition is looked at now, and again each time a response or an
    unsolicited response has been taken into the store and
    :attr:`indications`, and when the connection is made or ends. It may
    look at anything, this outstation included:

    .. code-block:: python

        await lab.wait_for(lambda store: store.analog_output(87).value == 5000, 10)
        await lab.wait_for(lambda _: lab.indications.is_set(IINBit.NEED_TIME), 10)

    Nothing is sent to make it come true. A timeout is a result and not an
    exception; an exception the condition raises is raised here.
    """
    if not (timeout >= 0 and math.isfinite(timeout)):
        raise ValueError(f"timeout is {timeout}; it is seconds, zero or more")
    loop = asyncio.get_running_loop()
    deadline = loop.time() + timeout
    while True:
        # Taken before the look, so a change during it is not missed.
        changed = self._changed
        if condition(self.store):
            return True
        remaining = deadline - loop.time()
        if remaining <= 0:
            return False
        with contextlib.suppress(TimeoutError):
            await asyncio.wait_for(changed.wait(), remaining)

repeat_scan ¶

repeat_scan(kind: str, interval: float | None) -> None

Repeat a scan every interval seconds, or stop repeating it with None.

No scan is repeated unless this is called. A repeated scan waits its turn like any other request, stops when the connection ends, and starts again when it is made again.

Source code in src/py1815/master/api.py

def repeat_scan(self, kind: str, interval: float | None) -> None:
    """Repeat a scan every ``interval`` seconds, or stop repeating it with None.

    No scan is repeated unless this is called. A repeated scan waits its
    turn like any other request,
    stops when the connection ends, and starts again when it is made again.
    """
    requests.scan(kind)  # Refuses a kind that is not a scan, before anything runs.
    running = self._scanning.pop(kind, None)
    if running is not None:
        running.cancel()
    if interval is None:
        self._scan_intervals.pop(kind, None)
        return
    if not interval > 0:
        raise ValueError(f"an interval of {interval} is not a wait between scans")
    self._scan_intervals[kind] = interval
    if self.connected:
        self._start_scan(kind, interval)

Master ¶

Outstations, by the names their caller gave them.

Source code in src/py1815/master/api.py

class Master:
    """Outstations, by the names their caller gave them."""

    def __init__(self) -> None:
        self._outstations: dict[str, Outstation] = {}

    async def __aenter__(self) -> Master:
        return self

    async def __aexit__(
        self,
        exc_type: type[BaseException] | None,
        exc: BaseException | None,
        traceback: TracebackType | None,
    ) -> None:
        await self.close()

    @property
    def outstations(self) -> dict[str, Outstation]:
        """The outstations added, by name."""
        return dict(self._outstations)

    def __getitem__(self, name: str) -> Outstation:
        return self._outstations[name]

    async def add(
        self, name: str, *, connect: bool = True, wait: float | None = None, **options: Any
    ) -> Outstation:
        """Add an outstation and, unless told not to, connect to it.

        ``options`` are those of :class:`Outstation`, and ``wait`` is that of
        :meth:`Outstation.connect`. A name is used once. An outstation whose
        connection cannot be made is not added.
        """
        if name in self._outstations:
            raise ValueError(f"an outstation named {name!r} has already been added")
        outstation = Outstation(name, **options)
        if connect:
            await outstation.connect(wait=wait)
        self._outstations[name] = outstation
        return outstation

    async def remove(self, name: str) -> None:
        """Close an outstation's connection and forget it."""
        outstation = self._outstations.pop(name)
        await outstation.close()

    async def close(self) -> None:
        """Close every connection."""
        outstations, self._outstations = list(self._outstations.values()), {}
        for outstation in outstations:
            await outstation.close()

outstations `property` ¶

outstations: dict[str, Outstation]

The outstations added, by name.

add `async` ¶

add(
    name: str,
    *,
    connect: bool = True,
    wait: float | None = None,
    **options: Any,
) -> Outstation

Add an outstation and, unless told not to, connect to it.

options are those of :class:Outstation, and wait is that of :meth:Outstation.connect. A name is used once. An outstation whose connection cannot be made is not added.

Source code in src/py1815/master/api.py

async def add(
    self, name: str, *, connect: bool = True, wait: float | None = None, **options: Any
) -> Outstation:
    """Add an outstation and, unless told not to, connect to it.

    ``options`` are those of :class:`Outstation`, and ``wait`` is that of
    :meth:`Outstation.connect`. A name is used once. An outstation whose
    connection cannot be made is not added.
    """
    if name in self._outstations:
        raise ValueError(f"an outstation named {name!r} has already been added")
    outstation = Outstation(name, **options)
    if connect:
        await outstation.connect(wait=wait)
    self._outstations[name] = outstation
    return outstation

remove `async` ¶

remove(name: str) -> None

Close an outstation's connection and forget it.

Source code in src/py1815/master/api.py

async def remove(self, name: str) -> None:
    """Close an outstation's connection and forget it."""
    outstation = self._outstations.pop(name)
    await outstation.close()

close `async` ¶

close() -> None

Close every connection.

Source code in src/py1815/master/api.py

async def close(self) -> None:
    """Close every connection."""
    outstations, self._outstations = list(self._outstations.values()), {}
    for outstation in outstations:
        await outstation.close()

The operations¶

py1815.master.operations ¶

The operations a master offers, whatever carries them.

One list of operations with one implementation each, so that a master wired straight to a session and a master on a socket cannot drift apart. What differs between them is only how an exchange is carried out, which is the one method a subclass supplies; over a socket that method is a coroutine, and the operations return what it returns.

Steps ¶

Bases: Protocol

Requests decided one at a time, each from the answers to those before it.

A select and its operate, and a time synchronization, are carried out this way, so every carrier follows one rule for when the second request is sent.

Source code in src/py1815/master/operations.py

class Steps(Protocol):
    """Requests decided one at a time, each from the answers to those before it.

    A select and its operate, and a time synchronization, are carried out
    this way, so every carrier follows one rule for when the second request
    is sent.
    """

    def next(self, done: Sequence[Any]) -> tuple[FunctionCode, bytes] | None:
        """Return the request to make after those in ``done``, or None."""

    def result(self, done: Sequence[Any]) -> Any:
        """Return what the requests made amount to."""

next ¶

next(
    done: Sequence[Any],
) -> tuple[FunctionCode, bytes] | None

Return the request to make after those in done, or None.

Source code in src/py1815/master/operations.py

def next(self, done: Sequence[Any]) -> tuple[FunctionCode, bytes] | None:
    """Return the request to make after those in ``done``, or None."""

result ¶

result(done: Sequence[Any]) -> Any

Return what the requests made amount to.

Source code in src/py1815/master/operations.py

def result(self, done: Sequence[Any]) -> Any:
    """Return what the requests made amount to."""

Operations ¶

Bases: Generic[ResultT, OperatedT, SyncedT]

What a master can be asked to do.

Source code in src/py1815/master/operations.py

class Operations(Generic[ResultT, OperatedT, SyncedT]):
    """What a master can be asked to do."""

    def _exchange(
        self, function: FunctionCode, body: bytes, *, broadcast: link.Broadcast | None = None
    ) -> ResultT:
        raise NotImplementedError

    def _carry_out(self, plan: controls.Plan) -> OperatedT:
        """Make the requests a plan calls for, one after another with none between."""
        raise NotImplementedError

    def _synchronize(self, plan: TimeSync) -> SyncedT:
        """Make the requests of a time synchronization, with none between them."""
        raise NotImplementedError

    def _now_ms(self) -> int:
        """Return the master's clock, in milliseconds since the epoch."""
        return round(time.time() * 1000)

    def operate(
        self,
        *,
        binary_outputs: Mapping[Any, Any] | None = None,
        analog_outputs: Mapping[Any, Any] | None = None,
        mode: controls.Mode | str = controls.Mode.DIRECT,
        variation: int | None = None,
    ) -> OperatedT:
        """Command outputs, named by index, in one request.

        A binary output is given true or false for a latch, or an operation
        by name: ``pulse_on``, ``trip``, ``close``. An analog output is given
        a number. Binary outputs are sent first, and each in the order given.

        ``mode`` is ``direct`` for a direct operate, ``select`` for a select
        followed by an operate, and ``direct_no_ack`` for a direct operate the
        outstation does not answer. After a select the operate is sent only if
        the outstation echoed every control unchanged and accepted each one.

        The result has a status for each control, and says whether the
        outstation accepted them all, refused, or never said. Nothing is sent
        twice: a response that does not arrive is reported, and whether to try
        again is the caller's to decide.

        A whole number is sent as an integer and anything else as a float,
        unless ``variation`` says which. An outstation that scales its points
        takes an integer as the transmitted value and a float as the
        engineering one.
        """
        made = controls.commands(binary_outputs, analog_outputs, variation=variation)
        return self._carry_out(controls.Plan(made, mode))

    def write_time(self, milliseconds: int | None = None) -> ResultT:
        """Set the outstation's clock: milliseconds since the epoch, UTC, or now."""
        when = self._now_ms() if milliseconds is None else int(milliseconds)
        return self._exchange(FunctionCode.WRITE, requests.write_time(when))

    def synchronize_time(self, procedure: str = LAN) -> SyncedT:
        """Set the outstation's clock by one of the procedures of IEEE 1815-2012 10.3.3.

        ``lan`` records the current time at the outstation and then writes
        the time the master sent that request. ``non_lan`` measures the delay
        and writes the master's time plus the delay. The write is sent only
        if the first request was answered without an error indication, and
        neither request is ever sent again. See :mod:`py1815.master.timesync`.
        """
        return self._synchronize(TimeSync(procedure, clock_ms=self._now_ms))

    def broadcast(
        self,
        function: FunctionCode,
        body: bytes = b"",
        *,
        address: link.Broadcast | int | str = DEFAULT_BROADCAST,
    ) -> ResultT:
        """Send a request to a broadcast address: every outstation on the link.

        ``address`` is ``no_confirm`` (0xFFFD), ``shall_confirm`` (0xFFFE) or
        ``optional_confirm`` (0xFFFF), which say whether an outstation asks
        for its next response to be confirmed. No outstation answers a
        broadcast, so the exchange ends as sent and the indication that one
        arrived (IIN1.0) comes in the next response to a request of this
        master's own. Nothing is sent again.
        """
        return self._exchange(function, body, broadcast=broadcast_address(address))

    def clear_restart(self) -> ResultT:
        """Clear the restart indication, which a master does once it has seen it."""
        return self._exchange(FunctionCode.WRITE, requests.clear_restart())

    def freeze(self, *, clear: bool = False, respond: bool = True) -> ResultT:
        """Freeze every counter, clearing each as it is frozen if asked.

        With ``respond`` false the request is one the outstation does not
        answer.
        """
        function = {
            (False, True): FunctionCode.IMMED_FREEZE,
            (False, False): FunctionCode.IMMED_FREEZE_NR,
            (True, True): FunctionCode.FREEZE_CLEAR,
            (True, False): FunctionCode.FREEZE_CLEAR_NR,
        }[(bool(clear), bool(respond))]
        return self._exchange(function, requests.freeze_counters())

    def restart(self, kind: str = "cold") -> ResultT:
        """Ask the outstation to restart: ``cold`` or ``warm``."""
        functions = {"cold": FunctionCode.COLD_RESTART, "warm": FunctionCode.WARM_RESTART}
        if kind not in functions:
            raise ValueError(f"{kind!r} is not a restart; cold or warm")
        return self._exchange(functions[kind], b"")

    def request(self, function: FunctionCode, body: bytes = b"") -> ResultT:
        """Send any request, as a function code and the octets after it.

        The escape hatch: whatever this interface does not name yet can still
        be asked, and the answer comes back decoded as far as it can be.
        """
        return self._exchange(function, body)

    def scan(self, kind: str) -> ResultT:
        """Read by name: ``integrity``, ``events``, ``class0`` to ``class3``, or ``outputs``.

        ``outputs`` reads binary and analog output status by their groups. It
        is not part of an integrity poll: the IEEE 1815.2 profile leaves
        output status out of class 0.
        """
        return self._exchange(FunctionCode.READ, requests.scan(kind))

    def integrity_poll(self) -> ResultT:
        """Read every event class and then every static value."""
        return self.scan("integrity")

    def enable_unsolicited(self, *classes: int) -> ResultT:
        """Ask the outstation to report the named event classes without being polled."""
        return self._exchange(
            FunctionCode.ENABLE_UNSOLICITED, requests.class_scan(*_event_classes(classes))
        )

    def disable_unsolicited(self, *classes: int) -> ResultT:
        """Ask the outstation to stop reporting the named event classes unasked."""
        return self._exchange(
            FunctionCode.DISABLE_UNSOLICITED, requests.class_scan(*_event_classes(classes))
        )

    def read(
        self,
        *,
        binary_inputs: Indices = None,
        binary_outputs: Indices = None,
        counters: Indices = None,
        frozen_counters: Indices = None,
        analog_inputs: Indices = None,
        analog_outputs: Indices = None,
    ) -> ResultT:
        """Read named points, in one request.

        Each argument is the indices wanted of one point type, or ``"all"``
        for every point of that type. The outstation answers in whichever
        variation it sends by default.
        """
        named = {
            PointType.BINARY_INPUT: binary_inputs,
            PointType.BINARY_OUTPUT: binary_outputs,
            PointType.COUNTER: counters,
            PointType.FROZEN_COUNTER: frozen_counters,
            PointType.ANALOG_INPUT: analog_inputs,
            PointType.ANALOG_OUTPUT: analog_outputs,
        }
        wanted: dict[PointType, Sequence[int] | None] = {}
        for point, indices in named.items():
            if indices is None:
                continue
            if isinstance(indices, str):
                if indices != ALL:
                    raise ValueError(f"{indices!r} is not a list of indices or 'all'")
                wanted[point] = None
            else:
                wanted[point] = list(indices)
        if not wanted:
            raise ValueError("a read names at least one point")
        return self._exchange(FunctionCode.READ, requests.read_points(wanted))

operate ¶

operate(
    *,
    binary_outputs: Mapping[Any, Any] | None = None,
    analog_outputs: Mapping[Any, Any] | None = None,
    mode: Mode | str = controls.Mode.DIRECT,
    variation: int | None = None,
) -> OperatedT

Command outputs, named by index, in one request.

A binary output is given true or false for a latch, or an operation by name: pulse_on, trip, close. An analog output is given a number. Binary outputs are sent first, and each in the order given.

mode is direct for a direct operate, select for a select followed by an operate, and direct_no_ack for a direct operate the outstation does not answer. After a select the operate is sent only if the outstation echoed every control unchanged and accepted each one.

The result has a status for each control, and says whether the outstation accepted them all, refused, or never said. Nothing is sent twice: a response that does not arrive is reported, and whether to try again is the caller's to decide.

A whole number is sent as an integer and anything else as a float, unless variation says which. An outstation that scales its points takes an integer as the transmitted value and a float as the engineering one.

Source code in src/py1815/master/operations.py

def operate(
    self,
    *,
    binary_outputs: Mapping[Any, Any] | None = None,
    analog_outputs: Mapping[Any, Any] | None = None,
    mode: controls.Mode | str = controls.Mode.DIRECT,
    variation: int | None = None,
) -> OperatedT:
    """Command outputs, named by index, in one request.

    A binary output is given true or false for a latch, or an operation
    by name: ``pulse_on``, ``trip``, ``close``. An analog output is given
    a number. Binary outputs are sent first, and each in the order given.

    ``mode`` is ``direct`` for a direct operate, ``select`` for a select
    followed by an operate, and ``direct_no_ack`` for a direct operate the
    outstation does not answer. After a select the operate is sent only if
    the outstation echoed every control unchanged and accepted each one.

    The result has a status for each control, and says whether the
    outstation accepted them all, refused, or never said. Nothing is sent
    twice: a response that does not arrive is reported, and whether to try
    again is the caller's to decide.

    A whole number is sent as an integer and anything else as a float,
    unless ``variation`` says which. An outstation that scales its points
    takes an integer as the transmitted value and a float as the
    engineering one.
    """
    made = controls.commands(binary_outputs, analog_outputs, variation=variation)
    return self._carry_out(controls.Plan(made, mode))

write_time ¶

write_time(milliseconds: int | None = None) -> ResultT

Set the outstation's clock: milliseconds since the epoch, UTC, or now.

Source code in src/py1815/master/operations.py

def write_time(self, milliseconds: int | None = None) -> ResultT:
    """Set the outstation's clock: milliseconds since the epoch, UTC, or now."""
    when = self._now_ms() if milliseconds is None else int(milliseconds)
    return self._exchange(FunctionCode.WRITE, requests.write_time(when))

synchronize_time ¶

synchronize_time(procedure: str = LAN) -> SyncedT

Set the outstation's clock by one of the procedures of IEEE 1815-2012 10.3.3.

lan records the current time at the outstation and then writes the time the master sent that request. non_lan measures the delay and writes the master's time plus the delay. The write is sent only if the first request was answered without an error indication, and neither request is ever sent again. See :mod:py1815.master.timesync.

Source code in src/py1815/master/operations.py

def synchronize_time(self, procedure: str = LAN) -> SyncedT:
    """Set the outstation's clock by one of the procedures of IEEE 1815-2012 10.3.3.

    ``lan`` records the current time at the outstation and then writes
    the time the master sent that request. ``non_lan`` measures the delay
    and writes the master's time plus the delay. The write is sent only
    if the first request was answered without an error indication, and
    neither request is ever sent again. See :mod:`py1815.master.timesync`.
    """
    return self._synchronize(TimeSync(procedure, clock_ms=self._now_ms))

broadcast ¶

broadcast(
    function: FunctionCode,
    body: bytes = b"",
    *,
    address: Broadcast | int | str = DEFAULT_BROADCAST,
) -> ResultT

Send a request to a broadcast address: every outstation on the link.

address is no_confirm (0xFFFD), shall_confirm (0xFFFE) or optional_confirm (0xFFFF), which say whether an outstation asks for its next response to be confirmed. No outstation answers a broadcast, so the exchange ends as sent and the indication that one arrived (IIN1.0) comes in the next response to a request of this master's own. Nothing is sent again.

Source code in src/py1815/master/operations.py

def broadcast(
    self,
    function: FunctionCode,
    body: bytes = b"",
    *,
    address: link.Broadcast | int | str = DEFAULT_BROADCAST,
) -> ResultT:
    """Send a request to a broadcast address: every outstation on the link.

    ``address`` is ``no_confirm`` (0xFFFD), ``shall_confirm`` (0xFFFE) or
    ``optional_confirm`` (0xFFFF), which say whether an outstation asks
    for its next response to be confirmed. No outstation answers a
    broadcast, so the exchange ends as sent and the indication that one
    arrived (IIN1.0) comes in the next response to a request of this
    master's own. Nothing is sent again.
    """
    return self._exchange(function, body, broadcast=broadcast_address(address))

clear_restart ¶

clear_restart() -> ResultT

Clear the restart indication, which a master does once it has seen it.

Source code in src/py1815/master/operations.py

def clear_restart(self) -> ResultT:
    """Clear the restart indication, which a master does once it has seen it."""
    return self._exchange(FunctionCode.WRITE, requests.clear_restart())

freeze ¶

freeze(
    *, clear: bool = False, respond: bool = True
) -> ResultT

Freeze every counter, clearing each as it is frozen if asked.

With respond false the request is one the outstation does not answer.

Source code in src/py1815/master/operations.py

def freeze(self, *, clear: bool = False, respond: bool = True) -> ResultT:
    """Freeze every counter, clearing each as it is frozen if asked.

    With ``respond`` false the request is one the outstation does not
    answer.
    """
    function = {
        (False, True): FunctionCode.IMMED_FREEZE,
        (False, False): FunctionCode.IMMED_FREEZE_NR,
        (True, True): FunctionCode.FREEZE_CLEAR,
        (True, False): FunctionCode.FREEZE_CLEAR_NR,
    }[(bool(clear), bool(respond))]
    return self._exchange(function, requests.freeze_counters())

restart ¶

restart(kind: str = 'cold') -> ResultT

Ask the outstation to restart: cold or warm.

Source code in src/py1815/master/operations.py

def restart(self, kind: str = "cold") -> ResultT:
    """Ask the outstation to restart: ``cold`` or ``warm``."""
    functions = {"cold": FunctionCode.COLD_RESTART, "warm": FunctionCode.WARM_RESTART}
    if kind not in functions:
        raise ValueError(f"{kind!r} is not a restart; cold or warm")
    return self._exchange(functions[kind], b"")

request ¶

request(
    function: FunctionCode, body: bytes = b""
) -> ResultT

Send any request, as a function code and the octets after it.

The escape hatch: whatever this interface does not name yet can still be asked, and the answer comes back decoded as far as it can be.

Source code in src/py1815/master/operations.py

def request(self, function: FunctionCode, body: bytes = b"") -> ResultT:
    """Send any request, as a function code and the octets after it.

    The escape hatch: whatever this interface does not name yet can still
    be asked, and the answer comes back decoded as far as it can be.
    """
    return self._exchange(function, body)

scan ¶

scan(kind: str) -> ResultT

Read by name: integrity, events, class0 to class3, or outputs.

outputs reads binary and analog output status by their groups. It is not part of an integrity poll: the IEEE 1815.2 profile leaves output status out of class 0.

Source code in src/py1815/master/operations.py

def scan(self, kind: str) -> ResultT:
    """Read by name: ``integrity``, ``events``, ``class0`` to ``class3``, or ``outputs``.

    ``outputs`` reads binary and analog output status by their groups. It
    is not part of an integrity poll: the IEEE 1815.2 profile leaves
    output status out of class 0.
    """
    return self._exchange(FunctionCode.READ, requests.scan(kind))

integrity_poll ¶

integrity_poll() -> ResultT

Read every event class and then every static value.

Source code in src/py1815/master/operations.py

def integrity_poll(self) -> ResultT:
    """Read every event class and then every static value."""
    return self.scan("integrity")

enable_unsolicited ¶

enable_unsolicited(*classes: int) -> ResultT

Ask the outstation to report the named event classes without being polled.

Source code in src/py1815/master/operations.py

def enable_unsolicited(self, *classes: int) -> ResultT:
    """Ask the outstation to report the named event classes without being polled."""
    return self._exchange(
        FunctionCode.ENABLE_UNSOLICITED, requests.class_scan(*_event_classes(classes))
    )

disable_unsolicited ¶

disable_unsolicited(*classes: int) -> ResultT

Ask the outstation to stop reporting the named event classes unasked.

Source code in src/py1815/master/operations.py

def disable_unsolicited(self, *classes: int) -> ResultT:
    """Ask the outstation to stop reporting the named event classes unasked."""
    return self._exchange(
        FunctionCode.DISABLE_UNSOLICITED, requests.class_scan(*_event_classes(classes))
    )

read ¶

read(
    *,
    binary_inputs: Indices = None,
    binary_outputs: Indices = None,
    counters: Indices = None,
    frozen_counters: Indices = None,
    analog_inputs: Indices = None,
    analog_outputs: Indices = None,
) -> ResultT

Read named points, in one request.

Each argument is the indices wanted of one point type, or "all" for every point of that type. The outstation answers in whichever variation it sends by default.

Source code in src/py1815/master/operations.py

def read(
    self,
    *,
    binary_inputs: Indices = None,
    binary_outputs: Indices = None,
    counters: Indices = None,
    frozen_counters: Indices = None,
    analog_inputs: Indices = None,
    analog_outputs: Indices = None,
) -> ResultT:
    """Read named points, in one request.

    Each argument is the indices wanted of one point type, or ``"all"``
    for every point of that type. The outstation answers in whichever
    variation it sends by default.
    """
    named = {
        PointType.BINARY_INPUT: binary_inputs,
        PointType.BINARY_OUTPUT: binary_outputs,
        PointType.COUNTER: counters,
        PointType.FROZEN_COUNTER: frozen_counters,
        PointType.ANALOG_INPUT: analog_inputs,
        PointType.ANALOG_OUTPUT: analog_outputs,
    }
    wanted: dict[PointType, Sequence[int] | None] = {}
    for point, indices in named.items():
        if indices is None:
            continue
        if isinstance(indices, str):
            if indices != ALL:
                raise ValueError(f"{indices!r} is not a list of indices or 'all'")
            wanted[point] = None
        else:
            wanted[point] = list(indices)
    if not wanted:
        raise ValueError("a read names at least one point")
    return self._exchange(FunctionCode.READ, requests.read_points(wanted))

broadcast_address ¶

broadcast_address(
    address: Broadcast | int | str,
) -> link.Broadcast

Return a broadcast address given by name, by number or as itself.

Source code in src/py1815/master/operations.py

def broadcast_address(address: link.Broadcast | int | str) -> link.Broadcast:
    """Return a broadcast address given by name, by number or as itself."""
    if isinstance(address, str):
        try:
            return BROADCASTS[address]
        except KeyError:
            raise ValueError(
                f"{address!r} is not a broadcast address; one of {', '.join(BROADCASTS)}"
            ) from None
    try:
        return link.Broadcast(address)
    except ValueError:
        raise ValueError(f"{address!r} is not a broadcast address: 0xFFFD to 0xFFFF") from None

Commands¶

py1815.master.controls ¶

Commands on outputs: what is sent, in what order, and what came of it.

A control is the one request a master makes that it must not get wrong and must not repeat. So this module does three things and nothing else. It turns what a caller asked for into the objects that travel, refusing anything that cannot be carried before a frame is built. It says which request follows which: a select is followed by an operate only when the outstation echoed every control back unchanged and accepted each one. And it reads the answer into a status for each control.

Nothing here retries. A response that never came is reported as one that never came, and whether to send the control again is the caller's to decide, because only the caller knows whether operating twice is harmless.

Mode ¶

Bases: Enum

How a control is sent.

Source code in src/py1815/master/controls.py

class Mode(Enum):
    """How a control is sent."""

    #: Direct operate: one request, answered with a status for each control.
    DIRECT = "direct"
    #: Select, then operate: the outstation is asked whether it would, and
    #: told to only if it said yes to everything.
    SELECT = "select"
    #: Direct operate with no acknowledgment: one request, and no answer.
    DIRECT_NO_ACK = "direct_no_ack"

Command `dataclass` ¶

One control, as it will travel.

Source code in src/py1815/master/controls.py

@dataclass(frozen=True)
class Command:
    """One control, as it will travel."""

    point: PointType
    index: int
    control: ControlRelayOutputBlock | AnalogOutput

    @property
    def group(self) -> int:
        binary = isinstance(self.control, ControlRelayOutputBlock)
        return GROUP_BINARY_OUTPUT_COMMAND if binary else GROUP_ANALOG_OUTPUT_COMMAND

    @property
    def variation(self) -> int:
        return 1 if isinstance(self.control, ControlRelayOutputBlock) else self.control.variation

    @property
    def encoded(self) -> bytes:
        """The object's octets, with the status a request carries: none."""
        return encode_control(self.control)

encoded `property` ¶

encoded: bytes

The object's octets, with the status a request carries: none.

PointStatus `dataclass` ¶

What an outstation said about one control.

Source code in src/py1815/master/controls.py

@dataclass(frozen=True)
class PointStatus:
    """What an outstation said about one control."""

    command: Command
    #: The status it answered with, or None if the response did not carry
    #: this control back at all.
    status: CommandStatus | None
    #: Whether what came back was the control as sent, apart from its status.
    #: A master is to check this: an echo that differs is not an answer to
    #: the request that was made.
    echoed: bool

    @property
    def accepted(self) -> bool:
        return self.echoed and self.status is CommandStatus.SUCCESS

Operated `dataclass` ¶

A control request, or the two of a select and operate, and what came of them.

Source code in src/py1815/master/controls.py

@dataclass(frozen=True)
class Operated:
    """A control request, or the two of a select and operate, and what came of them."""

    mode: Mode
    commands: tuple[Command, ...]
    #: Each exchange made, in order: one, or a select and then an operate.
    exchanges: tuple[Exchange, ...]
    #: Each control's status, from the last response that carried any.
    statuses: tuple[PointStatus, ...]

    @property
    def operated(self) -> bool:
        """Whether a request that operates was sent.

        False for a select the outstation refused, after which no operate is
        sent. True says only that it was sent, not that it was acted on.
        """
        return bool(self.exchanges) and self.exchanges[-1].function is not FunctionCode.SELECT

    @property
    def accepted(self) -> bool | None:
        """Whether the outstation accepted every control: yes, no, or not known.

        None when it never said: the request takes no acknowledgment, or the
        response did not arrive. That is not a refusal, and the output may
        have been operated.
        """
        if not self.operated:
            return False
        last = self.exchanges[-1]
        if not last.complete:
            return None
        return all(status.accepted for status in self.statuses)

    def status(self, point: PointType | str, index: int) -> CommandStatus | None:
        """The status of the control on one output, or None if there was none."""
        wanted = PointType(point)
        for status in self.statuses:
            if status.command.point is wanted and status.command.index == index:
                return status.status
        return None

operated `property` ¶

operated: bool

Whether a request that operates was sent.

False for a select the outstation refused, after which no operate is sent. True says only that it was sent, not that it was acted on.

accepted `property` ¶

accepted: bool | None

Whether the outstation accepted every control: yes, no, or not known.

None when it never said: the request takes no acknowledgment, or the response did not arrive. That is not a refusal, and the output may have been operated.

status ¶

status(
    point: PointType | str, index: int
) -> CommandStatus | None

The status of the control on one output, or None if there was none.

Source code in src/py1815/master/controls.py

def status(self, point: PointType | str, index: int) -> CommandStatus | None:
    """The status of the control on one output, or None if there was none."""
    wanted = PointType(point)
    for status in self.statuses:
        if status.command.point is wanted and status.command.index == index:
            return status.status
    return None

Plan ¶

The requests of one operation, decided one at a time.

Whatever carries the requests asks what to send next and is told, so a master on a socket and one wired to a session follow the same rule for when a select is followed by an operate.

Source code in src/py1815/master/controls.py

class Plan:
    """The requests of one operation, decided one at a time.

    Whatever carries the requests asks what to send next and is told, so a
    master on a socket and one wired to a session follow the same rule for
    when a select is followed by an operate.
    """

    def __init__(self, controls: Sequence[Command], mode: Mode | str = Mode.DIRECT) -> None:
        if not controls:
            raise ValueError("an operate names at least one output")
        try:
            self.mode = Mode(mode)
        except ValueError:
            names = ", ".join(each.value for each in Mode)
            raise ValueError(f"{mode!r} is not a way to operate; one of {names}") from None
        self.commands = tuple(controls)
        self.body = encode(self.commands)

    def next(self, done: Sequence[Exchange]) -> tuple[FunctionCode, bytes] | None:
        """The request to make after those already made, or None when there is none."""
        if not done:
            first = {
                Mode.DIRECT: FunctionCode.DIRECT_OPERATE,
                Mode.SELECT: FunctionCode.SELECT,
                Mode.DIRECT_NO_ACK: FunctionCode.DIRECT_OPERATE_NR,
            }[self.mode]
            return first, self.body
        if self.mode is Mode.SELECT and len(done) == 1 and self._selected(done[0]):
            return FunctionCode.OPERATE, self.body
        return None

    def _selected(self, select: Exchange) -> bool:
        return select.complete and all(
            status.accepted for status in read_statuses(self.commands, select)
        )

    def result(self, done: Sequence[Exchange]) -> Operated:
        """What the exchanges made amount to."""
        statuses: tuple[PointStatus, ...] = tuple(
            PointStatus(command, None, False) for command in self.commands
        )
        for exchange in reversed(done):
            if exchange.fragments:
                statuses = read_statuses(self.commands, exchange)
                break
        return Operated(self.mode, self.commands, tuple(done), statuses)

next ¶

next(
    done: Sequence[Exchange],
) -> tuple[FunctionCode, bytes] | None

The request to make after those already made, or None when there is none.

Source code in src/py1815/master/controls.py

def next(self, done: Sequence[Exchange]) -> tuple[FunctionCode, bytes] | None:
    """The request to make after those already made, or None when there is none."""
    if not done:
        first = {
            Mode.DIRECT: FunctionCode.DIRECT_OPERATE,
            Mode.SELECT: FunctionCode.SELECT,
            Mode.DIRECT_NO_ACK: FunctionCode.DIRECT_OPERATE_NR,
        }[self.mode]
        return first, self.body
    if self.mode is Mode.SELECT and len(done) == 1 and self._selected(done[0]):
        return FunctionCode.OPERATE, self.body
    return None

result ¶

result(done: Sequence[Exchange]) -> Operated

What the exchanges made amount to.

Source code in src/py1815/master/controls.py

def result(self, done: Sequence[Exchange]) -> Operated:
    """What the exchanges made amount to."""
    statuses: tuple[PointStatus, ...] = tuple(
        PointStatus(command, None, False) for command in self.commands
    )
    for exchange in reversed(done):
        if exchange.fragments:
            statuses = read_statuses(self.commands, exchange)
            break
    return Operated(self.mode, self.commands, tuple(done), statuses)

binary ¶

binary(index: Any, value: Any) -> Command

A command on a binary output.

value is true or false for a latch on or off, one of the names in :data:OPERATIONS, a mapping with an operation and optionally a count, on_time_ms and off_time_ms, or a control relay output block made elsewhere.

Source code in src/py1815/master/controls.py

def binary(index: Any, value: Any) -> Command:
    """A command on a binary output.

    ``value`` is true or false for a latch on or off, one of the names in
    :data:`OPERATIONS`, a mapping with an ``operation`` and optionally a
    ``count``, ``on_time_ms`` and ``off_time_ms``, or a control relay output
    block made elsewhere.
    """
    if isinstance(value, ControlRelayOutputBlock):
        return Command(PointType.BINARY_OUTPUT, _index(index), value)
    options: dict[str, int] = {}
    if isinstance(value, Mapping):
        unknown = set(value) - {"operation", "count", "on_time_ms", "off_time_ms"}
        if unknown:
            raise ValueError(f"a binary output command has no {sorted(unknown)[0]!r}")
        options = {name: int(value[name]) for name in value if name != "operation"}
        value = value.get("operation")
    if isinstance(value, bool):
        value = "latch_on" if value else "latch_off"
    if not isinstance(value, str) or value.lower() not in OPERATIONS:
        raise ValueError(
            f"{value!r} is not something a binary output can be told; true, false, "
            f"or one of {', '.join(OPERATIONS)}"
        )
    operation, trip_close = OPERATIONS[value.lower()]
    try:
        block = ControlRelayOutputBlock.build(
            operation,
            trip_close=trip_close,
            count=options.get("count", 1),
            on_time_ms=options.get("on_time_ms", 0),
            off_time_ms=options.get("off_time_ms", 0),
        )
        command = Command(PointType.BINARY_OUTPUT, _index(index), block)
        _ = command.encoded
    except (ControlError, struct.error) as error:
        raise ValueError(f"binary output {index}: {error}") from None
    return command

analog ¶

analog(
    index: Any, value: Any, variation: int | None = None
) -> Command

A command on an analog output.

value is a number, a numeric string, or an analog output made elsewhere. variation is 1 or 2 for a 32- or 16-bit integer and 3 or 4 for a single or double float; left out, the narrowest that carries the value is used.

Source code in src/py1815/master/controls.py

def analog(index: Any, value: Any, variation: int | None = None) -> Command:
    """A command on an analog output.

    ``value`` is a number, a numeric string, or an analog output made
    elsewhere. ``variation`` is 1 or 2 for a 32- or 16-bit integer and 3 or 4
    for a single or double float; left out, the narrowest that carries the
    value is used.
    """
    if isinstance(value, AnalogOutput):
        return Command(PointType.ANALOG_OUTPUT, _index(index), value)
    if isinstance(value, bool):
        raise ValueError(f"analog output {index} takes a number, not {value!r}")
    try:
        number = float(value)
    except (TypeError, ValueError):
        raise ValueError(f"analog output {index} takes a number, not {value!r}") from None
    chosen = _variation_for(number) if variation is None else int(variation)
    try:
        command = Command(PointType.ANALOG_OUTPUT, _index(index), AnalogOutput(number, chosen))
        _ = command.encoded
    except ControlError as error:
        raise ValueError(f"analog output {index}: {error}") from None
    return command

commands ¶

commands(
    binary_outputs: Mapping[Any, Any] | None = None,
    analog_outputs: Mapping[Any, Any] | None = None,
    *,
    variation: int | None = None,
) -> tuple[Command, ...]

The commands for outputs named by index: binary first, each in the order given.

Source code in src/py1815/master/controls.py

def commands(
    binary_outputs: Mapping[Any, Any] | None = None,
    analog_outputs: Mapping[Any, Any] | None = None,
    *,
    variation: int | None = None,
) -> tuple[Command, ...]:
    """The commands for outputs named by index: binary first, each in the order given."""
    made = [binary(index, value) for index, value in (binary_outputs or {}).items()]
    made += [analog(index, value, variation) for index, value in (analog_outputs or {}).items()]
    if not made:
        raise ValueError("an operate names at least one output")
    return tuple(made)

encode ¶

encode(controls: Sequence[Command]) -> bytes

The object headers and objects of a control request, in the order given.

Controls of one group and variation that stand together share a header. Each object is prefixed by its index, in one octet where every index of the header fits and in two where one does not.

Source code in src/py1815/master/controls.py

def encode(controls: Sequence[Command]) -> bytes:
    """The object headers and objects of a control request, in the order given.

    Controls of one group and variation that stand together share a header.
    Each object is prefixed by its index, in one octet where every index of
    the header fits and in two where one does not.
    """
    body = bytearray()
    for (group, variation), run in groupby(controls, key=lambda c: (c.group, c.variation)):
        together = list(run)
        narrow = len(together) <= 0xFF and all(command.index <= 0xFF for command in together)
        if narrow:
            body += bytes([group, variation, QualifierCode.UINT8_COUNT_UINT8_INDEX, len(together)])
        else:
            body += struct.pack(
                "<BBBH", group, variation, QualifierCode.UINT16_COUNT_UINT16_INDEX, len(together)
            )
        for command in together:
            body += struct.pack("<B" if narrow else "<H", command.index) + command.encoded
    return bytes(body)

read_statuses ¶

read_statuses(
    controls: Sequence[Command], exchange: Exchange
) -> tuple[PointStatus, ...]

Each control's status, from the objects a response echoed.

Source code in src/py1815/master/controls.py

def read_statuses(controls: Sequence[Command], exchange: Exchange) -> tuple[PointStatus, ...]:
    """Each control's status, from the objects a response echoed."""
    echoes = [
        decoded
        for decoded in exchange.objects
        if decoded.group in (GROUP_BINARY_OUTPUT_COMMAND, GROUP_ANALOG_OUTPUT_COMMAND)
    ]
    statuses = []
    for command in controls:
        match = next(
            (
                echo
                for echo in echoes
                if (echo.group, echo.variation, echo.index)
                == (command.group, command.variation, command.index)
            ),
            None,
        )
        if match is None:
            statuses.append(PointStatus(command, None, False))
            continue
        # An index may be commanded twice in one request, and each echo
        # answers one of them, in order.
        echoes.remove(match)
        try:
            status = decode_control(match.group, match.variation, match.raw).status
        except ControlError:
            statuses.append(PointStatus(command, None, False))
            continue
        statuses.append(PointStatus(command, status, match.raw[:-1] == command.encoded[:-1]))
    return tuple(statuses)

An IEEE 1815.2 DER¶

py1815.master.profile ¶

The IEEE 1815.2 DER profile, from the master's side.

py1815.profile resolves the profile's tables into a point map: each point's name, units, multiplier, range and section, which output enables a function and which input says it is supported. :class:DerProfile reads that map the other way round, for a master: it finds a point by its name, groups points by what they are for, and converts between engineering units and the numbers that travel. :class:Der puts it in front of an outstation, and :class:LoopbackDer in front of a :class:~py1815.master.loopback.Loopback.

Every operation is made of the master's own requests: a read is read, and a write is operate. A write is a control, so it is sent once and never repeated (D80): a write whose answer did not arrive is reported as not known, and nothing follows it that depends on it.

Each operation is written once, as a plan: a generator that yields one request at a time and is sent what came of it. A master on a socket awaits each request and one wired to a session does not, so the two cannot come to differ.

Function `dataclass` ¶

One DER function, as the tables group its points.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Function:
    """One DER function, as the tables group its points."""

    #: The function's name in lower case with hyphens, as in ``volt-var-control``.
    key: str
    #: The function's name, from the output that enables it.
    name: str
    #: What the tables say its points are for, as in ``Volt-Var``.
    purpose: str
    #: The binary output that enables it.
    enable: Point
    #: The binary input that reports whether it is enabled, when the tables pair one.
    status: Point | None
    #: The binary input that says whether the outstation supports it.
    supports: Point
    #: Its other outputs: settings, each read back by the input it is paired with.
    settings: tuple[Point, ...]
    #: Its other inputs.
    inputs: tuple[Point, ...]

    @property
    def points(self) -> tuple[Point, ...]:
        """Every point of the function, outputs first, each kind in index order."""
        every = {self.enable, self.supports, *self.settings, *self.inputs}
        if self.status is not None:
            every.add(self.status)
        return tuple(sorted(every, key=_by_address))

    @property
    def curve_settings(self) -> tuple[Point, ...]:
        """The settings that name a curve by its number."""
        return tuple(
            point
            for point in self.settings
            if point.kind is Kind.AO and "curve index" in point.name.casefold()
        )

points `property` ¶

points: tuple[Point, ...]

Every point of the function, outputs first, each kind in index order.

curve_settings `property` ¶

curve_settings: tuple[Point, ...]

The settings that name a curve by its number.

Reading `dataclass` ¶

What an outstation reported for one point of the profile.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Reading:
    """What an outstation reported for one point of the profile."""

    point: Point
    #: The value as it travelled: a state, a count, or a transmitted number.
    #: None when the outstation did not report the point.
    raw: bool | int | float | None
    flags: int | None
    time_ms: int | None
    reported: bool

    @property
    def value(self) -> bool | int | float | None:
        """The value in engineering units: the transmitted number through the multiplier.

        A whole transmitted number is given to as many decimal places as the
        multiplier and offset have, so 4224 tenths of a volt reads 422.4 and
        not the 422.40000000000003 that binary floating point makes of it.
        """
        if self.raw is None or isinstance(self.raw, bool):
            return self.raw
        value = self.point.from_wire(self.raw)
        if isinstance(self.raw, int) and math.isfinite(value):
            return round(value, _places(self.point.multiplier) + _places(self.point.offset))
        return value

    @property
    def state(self) -> str | None:
        """A binary point's state by the name the tables give it."""
        if not isinstance(self.raw, bool) or self.point.states is None:
            return None
        return self.point.states[int(self.raw)]

    @property
    def quality(self) -> str:
        """What its flags say of it. See :func:`quality`."""
        return quality(self.flags, self.reported)

value `property` ¶

value: bool | int | float | None

The value in engineering units: the transmitted number through the multiplier.

A whole transmitted number is given to as many decimal places as the multiplier and offset have, so 4224 tenths of a volt reads 422.4 and not the 422.40000000000003 that binary floating point makes of it.

state `property` ¶

state: str | None

A binary point's state by the name the tables give it.

quality `property` ¶

quality: str

What its flags say of it. See :func:quality.

Readings `dataclass` ¶

Points read in one request, in the order they were asked for.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Readings:
    """Points read in one request, in the order they were asked for."""

    readings: tuple[Reading, ...]
    #: The read, and each range read again when the outstation refused it whole.
    exchanges: tuple[Exchange, ...]

    def __getitem__(self, name: str) -> Reading:
        """Return one reading, by the point's address or its name."""
        for reading in self.readings:
            if name.strip().upper() == address(reading.point) or _normal(name) in (
                _normal(reading.point.name),
                _normal(label(reading.point)),
            ):
                return reading
        raise KeyError(f"{name!r} was not read")

Setpoint `dataclass` ¶

One output written, and what came of it.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Setpoint:
    """One output written, and what came of it."""

    point: Point
    #: The value asked for, in engineering units, or the state of a binary output.
    requested: bool | float
    #: What travelled: a transmitted number, or the engineering value where a
    #: floating-point variation was asked for, or the state of a binary output.
    sent: bool | int | float
    status: controls.PointStatus
    #: The input that mirrors the output, read afterwards, when ``verify`` asked.
    readback: Reading | None = None

    @property
    def sent_value(self) -> bool | float:
        """What was sent, in engineering units."""
        if isinstance(self.sent, (bool, float)):
            return self.sent
        return self.point.from_wire(self.sent)

    @property
    def matches(self) -> bool | None:
        """Whether the readback is what was asked for, within one step of the multiplier.

        None when there was no readback, or the outstation did not report it.
        """
        if self.readback is None or not self.readback.reported:
            return None
        read = self.readback.value
        if isinstance(self.requested, bool) or isinstance(read, bool):
            return bool(read) == bool(self.requested)
        if read is None or not math.isfinite(float(read)):
            return False
        step = abs(self.point.multiplier) if self.point.multiplier else 1.0
        return abs(float(read) - float(self.requested)) <= step * (1 + 1e-9)

sent_value `property` ¶

sent_value: bool | float

What was sent, in engineering units.

matches `property` ¶

matches: bool | None

Whether the readback is what was asked for, within one step of the multiplier.

None when there was no readback, or the outstation did not report it.

Written `dataclass` ¶

A write of outputs by name, and, when asked, what their mirrors read afterwards.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Written:
    """A write of outputs by name, and, when asked, what their mirrors read afterwards."""

    operated: Operated
    setpoints: tuple[Setpoint, ...]
    #: The read of the mirroring inputs, when ``verify`` asked for one.
    verification: Readings | None = None

    @property
    def accepted(self) -> bool | None:
        """Whether the outstation accepted every write: True, False, or None when not known."""
        return self.operated.accepted

    @property
    def verified(self) -> bool | None:
        """Whether every readback matched: True, False if one did not, None if not known."""
        if self.verification is None:
            return None
        matches = [setpoint.matches for setpoint in self.setpoints]
        if any(match is False for match in matches):
            return False
        if matches and all(match is True for match in matches):
            return True
        return None

accepted `property` ¶

accepted: bool | None

Whether the outstation accepted every write: True, False, or None when not known.

verified `property` ¶

verified: bool | None

Whether every readback matched: True, False if one did not, None if not known.

Switched `dataclass` ¶

A function enabled or disabled, and what its status input then said.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Switched:
    """A function enabled or disabled, and what its status input then said."""

    function: Function
    #: True for an enable, False for a disable.
    enable: bool
    operated: Operated
    #: The input that reports whether the function is enabled, read afterwards.
    status: Reading | None
    #: The read of that input, or None where the tables pair none.
    readback: Readings | None

    @property
    def accepted(self) -> bool | None:
        return self.operated.accepted

    @property
    def enabled(self) -> bool | None:
        """Whether the outstation says the function is enabled. None when it did not say."""
        if self.status is None or not self.status.reported:
            return None
        return bool(self.status.raw)

enabled `property` ¶

enabled: bool | None

Whether the outstation says the function is enabled. None when it did not say.

Curve `dataclass` ¶

The curve the curve block shows, as the outstation reported it.

The points are the numbers that travelled: their scaling depends on the units the curve declares, and the tables give the block no multiplier.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Curve:
    """The curve the curve block shows, as the outstation reported it.

    The points are the numbers that travelled: their scaling depends on the
    units the curve declares, and the tables give the block no multiplier.
    """

    number: int | None
    type: int | None
    count: int | None
    x_units: int | None
    y_units: int | None
    points: tuple[tuple[float, float], ...]
    #: Whether a function names this curve, where the outstation says.
    referenced: bool | None
    #: The reads it took: the selector and fields, then the points.
    reads: tuple[Readings, ...]
    #: The readings of the selector and the four fields, in that order.
    fields: tuple[Reading, ...] = ()

CurveWritten `dataclass` ¶

A curve written as the profile requires, and read back.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class CurveWritten:
    """A curve written as the profile requires, and read back."""

    number: int
    type: int
    x_units: int
    y_units: int
    points: tuple[tuple[int, int], ...]
    #: Each write made, in order: the selector, the fields, the points. A
    #: write is made only when the one before it was accepted.
    steps: tuple[Operated, ...]
    readback: Curve

    @property
    def stopped_at(self) -> str | None:
        """The step that was not accepted, by name, or None when none failed."""
        names = ("selector", "fields", "points")
        for name, step in zip(names, self.steps, strict=False):
            if step.accepted is not True:
                return name
        return None

    @property
    def accepted(self) -> bool | None:
        """True when every write was accepted, False when one was refused, None when not known."""
        for step in self.steps:
            if step.accepted is not True:
                return step.accepted
        return True

    @property
    def matches(self) -> bool:
        """Whether the curve read back is the curve written."""
        back = self.readback
        return (
            back.number == self.number
            and back.type == self.type
            and back.count == len(self.points)
            and back.x_units == self.x_units
            and back.y_units == self.y_units
            and tuple((float(x), float(y)) for x, y in self.points) == back.points
        )

stopped_at `property` ¶

stopped_at: str | None

The step that was not accepted, by name, or None when none failed.

accepted `property` ¶

accepted: bool | None

True when every write was accepted, False when one was refused, None when not known.

matches `property` ¶

matches: bool

Whether the curve read back is the curve written.

FunctionState `dataclass` ¶

One function, and what the outstation says of it.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class FunctionState:
    """One function, and what the outstation says of it."""

    function: Function
    #: Whether the outstation says it supports the function. None when it did not say.
    supported: bool | None
    #: Whether the outstation says it is enabled. None when it did not say.
    enabled: bool | None

Functions `dataclass` ¶

Which functions an outstation supports and which are enabled.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Functions:
    """Which functions an outstation supports and which are enabled."""

    states: tuple[FunctionState, ...]
    readings: Readings

Declared `dataclass` ¶

One point as a Device Profile document declares it.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Declared:
    """One point as a Device Profile document declares it."""

    type: PointType
    index: int
    name: str | None
    #: The class its events are reported in: 1, 2 or 3, 0 for none, None where not stated.
    event_class: int | None
    #: Whether a class 0 read carries it. None where not stated.
    class_0: bool | None
    #: Its deadband, in transmitted units, where stated.
    deadband: float | None

Comparison `dataclass` ¶

What an outstation serves, against what its Device Profile document declares.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Comparison:
    """What an outstation serves, against what its Device Profile document declares."""

    declared: tuple[Declared, ...]
    #: Declared, and not served: no read returned it.
    absent: tuple[Declared, ...]
    #: Served, and not declared, by type and index.
    undeclared: tuple[tuple[PointType, int], ...]
    #: Served points whose presence in a class 0 read is not what is declared,
    #: each with whether the class 0 read carried it.
    class_0: tuple[tuple[Declared, bool], ...]
    #: Declared and served. A point's event class and deadband cannot be read
    #: from an outstation, so they are given as declared.
    served: tuple[Declared, ...]
    exchanges: tuple[Exchange, ...]

DerProfile ¶

The point map, read for a master: names, groups, functions and scaling. No I/O.

Source code in src/py1815/master/profile.py

class DerProfile:
    """The point map, read for a master: names, groups, functions and scaling. No I/O."""

    def __init__(self, point_map: PointMap, *, curve_selector: int = CURVE_SELECTOR) -> None:
        """
        Args:
            point_map: The profile's points, resolved for the outstation.
            curve_selector: The analog output that selects the curve the
                curve block shows. The fields and points follow it.
        """
        self.map = point_map
        self.functions: tuple[Function, ...] = self._functions()
        self._curve_selector = curve_selector

    # ---------------------------------------------------------------- names

    def point(self, name: str, *, outputs: bool = False) -> Point:
        """Return a point by its address, as in ``AO87``, or by its name.

        A name is matched whole or by its first sentence, ignoring case and
        spacing. The tables give some names to more than one point, an output
        and the input that reads it back among them, so a name is looked for
        among the inputs first and then among the outputs, or among the
        outputs only when ``outputs`` is set. A name that still matches more
        than one point is refused, with the addresses it could mean.
        """
        given = _ADDRESS.match(name)
        if given:
            kind, index = Kind(given.group(1).upper()), int(given.group(2))
            found = self.map.get(kind, index)
            if found is None:
                raise ValueError(f"the profile has no {kind.value}{index}")
            if outputs and not kind.is_output:
                raise ValueError(f"{kind.value}{index} is an input, and is not written")
            return found
        wanted = _normal(name)
        if not wanted:
            raise ValueError("a point is named by its address or its name")
        for output_tier in (True,) if outputs else (False, True):
            candidates = [p for p in self.map.points.values() if p.kind.is_output == output_tier]
            # The whole name first, then the first sentence of it.
            for whole in (True, False):
                named = [p for p in candidates if _normal(p.name if whole else label(p)) == wanted]
                if len(named) == 1:
                    return named[0]
                if len(named) > 1:
                    listed = ", ".join(address(p) for p in sorted(named, key=_by_address))
                    raise ValueError(f"{name!r} names more than one point: {listed}")
        raise ValueError(f"the profile has no point named {name!r}")

    def function(self, name: str) -> Function:
        """Return a function by its key, name, purpose, or the address of its enable output."""
        wanted = _normal(name)
        for function in self.functions:
            if wanted in (
                function.key,
                _normal(function.name),
                _normal(function.purpose),
                _normal(address(function.enable)),
            ):
                return function
        known = ", ".join(function.key for function in self.functions) or "none"
        raise ValueError(f"the profile has no function {name!r}; it has {known}")

    def group(self, name: str) -> tuple[Point, ...]:
        """Return the points of a named group: a function, or everything of one purpose.

        A purpose is what the tables say a point is for: ``Nameplate``,
        ``Monitoring``, ``Return to Service``. A function's group is its own
        points, which may be fewer than its purpose names elsewhere.
        """
        try:
            return self.function(name).points
        except ValueError:
            pass
        wanted = _normal(name)
        found = [p for p in self.map.points.values() if p.purpose and _normal(p.purpose) == wanted]
        if not found:
            raise ValueError(f"the profile has no function or group named {name!r}")
        return tuple(sorted(found, key=_by_address))

    def mirror(self, point: Point) -> Point | None:
        """Return the input that reads an output back, or None when the tables pair none."""
        paired = point.associated
        if paired is None or paired[0].is_output:
            return None
        return self.map.get(*paired)

    # -------------------------------------------------------------- scaling

    def transmitted(self, point: Point, value: Any) -> bool | int:
        """Return an output's value as it travels: a state, or a whole transmitted number.

        A binary output takes true or false, 1 or 0, or the name the tables
        give one of its states. An analog output takes a number in engineering
        units, divided by its multiplier and rounded to the nearest whole
        number. A value outside the point's range is refused here, before
        anything is sent.
        """
        where = f"{address(point)} ({label(point)})"
        if point.kind is Kind.BO:
            if isinstance(value, bool):
                return value
            if isinstance(value, int) and value in (0, 1):
                return bool(value)
            if isinstance(value, str) and point.states is not None:
                for state, text in enumerate(point.states):
                    if _normal(value) == _normal(text):
                        return bool(state)
            states = f", or {' or '.join(repr(s) for s in point.states)}" if point.states else ""
            raise ValueError(f"{where} takes true or false{states}, not {value!r}")
        if point.kind is not Kind.AO:
            raise ValueError(f"{where} is an input, and is not written")
        if isinstance(value, bool):
            raise ValueError(f"{where} takes a number, not {value!r}")
        try:
            number = float(value)
        except (TypeError, ValueError, OverflowError):
            # OverflowError: a JSON integer such as 10**1000 has no float.
            raise ValueError(f"{where} takes a number, not {value!r}") from None
        raw = point.to_wire(number)
        if not math.isfinite(raw):
            raise ValueError(f"{where} takes a finite number, not {value!r}")
        whole = round(raw)
        if not point.in_range(whole) or not _INT32[0] <= whole <= _INT32[1]:
            raise ValueError(f"{where}: {number:g} is outside {self.span(point)}")
        return whole

    @staticmethod
    def limits(point: Point) -> tuple[float | None, float | None]:
        """Return an analog point's range in engineering units, lowest first; None where open."""
        ends = [
            None if bound is None else point.from_wire(float(bound))
            for bound in (point.minimum, point.maximum)
        ]
        if ends[0] is not None and ends[1] is not None and ends[0] > ends[1]:
            ends.reverse()
        return ends[0], ends[1]

    @staticmethod
    def span(point: Point) -> str:
        """Return an analog point's range in engineering units, as text."""
        low, high = DerProfile.limits(point)
        units = f" {point.units}" if point.units and point.units.lower() not in ("n/a",) else ""
        lower = "no lower limit" if low is None else f"{low:g}"
        upper = "no upper limit" if high is None else f"{high:g}"
        return f"{lower} to {upper}{units}"

    # --------------------------------------------------------------- curves

    def curve_block(self) -> tuple[list[Point], list[Point]]:
        """Return the curve block's outputs and the inputs that read them back, in order.

        The selector, the four fields, then X and Y of each point. Raises
        ValueError when the map does not hold the block or its readback.
        """
        outputs: list[Point] = []
        inputs: list[Point] = []
        for offset in range(1 + curves.FIELDS + 2 * curves.MAX_POINTS):
            output = self.map.get(Kind.AO, self._curve_selector + offset)
            if output is None:
                raise ValueError(
                    f"the profile has no AO{self._curve_selector + offset}: no curve block "
                    f"starts at AO{self._curve_selector}"
                )
            readback = self.mirror(output)
            if readback is None:
                raise ValueError(f"the profile pairs no input with {address(output)}")
            outputs.append(output)
            inputs.append(readback)
        return outputs, inputs

    def referenced(self) -> Point | None:
        """Return the input that says whether a function names the selected curve, if any."""
        return self.map.get(Kind.BI, CURVE_REFERENCED)

    # ------------------------------------------------------------ functions

    def _functions(self) -> tuple[Function, ...]:
        """Find every function: a supports input and the enable output it is paired with.

        A function's points are the ones the tables give the enable output's
        purpose, under its heading, as the outstation groups them (D57).
        """
        found: list[Function] = []
        for supports in self.map.of(Kind.BI):
            if supports.enabled_by is None or supports.enabled_by not in self.map:
                continue
            enable = self.map.point(*supports.enabled_by)
            purpose = enable.purpose or ""
            status = self.mirror(enable)
            members = [
                p
                for p in self.map.points.values()
                if purpose
                and p.purpose
                and p.purpose.casefold() == purpose.casefold()
                and p.section == enable.section
                and p.address not in (enable.address, supports.address)
                and (status is None or p.address != status.address)
            ]
            ordered = sorted(members, key=_by_address)
            name = _function_name(enable)
            found.append(
                Function(
                    key=_key(name),
                    name=name,
                    purpose=purpose or name,
                    enable=enable,
                    status=status,
                    supports=supports,
                    settings=tuple(p for p in ordered if p.kind.is_output),
                    inputs=tuple(p for p in ordered if not p.kind.is_output),
                )
            )
        return tuple(found)

point ¶

point(name: str, *, outputs: bool = False) -> Point

Return a point by its address, as in AO87, or by its name.

A name is matched whole or by its first sentence, ignoring case and spacing. The tables give some names to more than one point, an output and the input that reads it back among them, so a name is looked for among the inputs first and then among the outputs, or among the outputs only when outputs is set. A name that still matches more than one point is refused, with the addresses it could mean.

Source code in src/py1815/master/profile.py

def point(self, name: str, *, outputs: bool = False) -> Point:
    """Return a point by its address, as in ``AO87``, or by its name.

    A name is matched whole or by its first sentence, ignoring case and
    spacing. The tables give some names to more than one point, an output
    and the input that reads it back among them, so a name is looked for
    among the inputs first and then among the outputs, or among the
    outputs only when ``outputs`` is set. A name that still matches more
    than one point is refused, with the addresses it could mean.
    """
    given = _ADDRESS.match(name)
    if given:
        kind, index = Kind(given.group(1).upper()), int(given.group(2))
        found = self.map.get(kind, index)
        if found is None:
            raise ValueError(f"the profile has no {kind.value}{index}")
        if outputs and not kind.is_output:
            raise ValueError(f"{kind.value}{index} is an input, and is not written")
        return found
    wanted = _normal(name)
    if not wanted:
        raise ValueError("a point is named by its address or its name")
    for output_tier in (True,) if outputs else (False, True):
        candidates = [p for p in self.map.points.values() if p.kind.is_output == output_tier]
        # The whole name first, then the first sentence of it.
        for whole in (True, False):
            named = [p for p in candidates if _normal(p.name if whole else label(p)) == wanted]
            if len(named) == 1:
                return named[0]
            if len(named) > 1:
                listed = ", ".join(address(p) for p in sorted(named, key=_by_address))
                raise ValueError(f"{name!r} names more than one point: {listed}")
    raise ValueError(f"the profile has no point named {name!r}")

function ¶

function(name: str) -> Function

Return a function by its key, name, purpose, or the address of its enable output.

Source code in src/py1815/master/profile.py

def function(self, name: str) -> Function:
    """Return a function by its key, name, purpose, or the address of its enable output."""
    wanted = _normal(name)
    for function in self.functions:
        if wanted in (
            function.key,
            _normal(function.name),
            _normal(function.purpose),
            _normal(address(function.enable)),
        ):
            return function
    known = ", ".join(function.key for function in self.functions) or "none"
    raise ValueError(f"the profile has no function {name!r}; it has {known}")

group ¶

group(name: str) -> tuple[Point, ...]

Return the points of a named group: a function, or everything of one purpose.

A purpose is what the tables say a point is for: Nameplate, Monitoring, Return to Service. A function's group is its own points, which may be fewer than its purpose names elsewhere.

Source code in src/py1815/master/profile.py

def group(self, name: str) -> tuple[Point, ...]:
    """Return the points of a named group: a function, or everything of one purpose.

    A purpose is what the tables say a point is for: ``Nameplate``,
    ``Monitoring``, ``Return to Service``. A function's group is its own
    points, which may be fewer than its purpose names elsewhere.
    """
    try:
        return self.function(name).points
    except ValueError:
        pass
    wanted = _normal(name)
    found = [p for p in self.map.points.values() if p.purpose and _normal(p.purpose) == wanted]
    if not found:
        raise ValueError(f"the profile has no function or group named {name!r}")
    return tuple(sorted(found, key=_by_address))

mirror ¶

mirror(point: Point) -> Point | None

Return the input that reads an output back, or None when the tables pair none.

Source code in src/py1815/master/profile.py

def mirror(self, point: Point) -> Point | None:
    """Return the input that reads an output back, or None when the tables pair none."""
    paired = point.associated
    if paired is None or paired[0].is_output:
        return None
    return self.map.get(*paired)

transmitted ¶

transmitted(point: Point, value: Any) -> bool | int

Return an output's value as it travels: a state, or a whole transmitted number.

A binary output takes true or false, 1 or 0, or the name the tables give one of its states. An analog output takes a number in engineering units, divided by its multiplier and rounded to the nearest whole number. A value outside the point's range is refused here, before anything is sent.

Source code in src/py1815/master/profile.py

def transmitted(self, point: Point, value: Any) -> bool | int:
    """Return an output's value as it travels: a state, or a whole transmitted number.

    A binary output takes true or false, 1 or 0, or the name the tables
    give one of its states. An analog output takes a number in engineering
    units, divided by its multiplier and rounded to the nearest whole
    number. A value outside the point's range is refused here, before
    anything is sent.
    """
    where = f"{address(point)} ({label(point)})"
    if point.kind is Kind.BO:
        if isinstance(value, bool):
            return value
        if isinstance(value, int) and value in (0, 1):
            return bool(value)
        if isinstance(value, str) and point.states is not None:
            for state, text in enumerate(point.states):
                if _normal(value) == _normal(text):
                    return bool(state)
        states = f", or {' or '.join(repr(s) for s in point.states)}" if point.states else ""
        raise ValueError(f"{where} takes true or false{states}, not {value!r}")
    if point.kind is not Kind.AO:
        raise ValueError(f"{where} is an input, and is not written")
    if isinstance(value, bool):
        raise ValueError(f"{where} takes a number, not {value!r}")
    try:
        number = float(value)
    except (TypeError, ValueError, OverflowError):
        # OverflowError: a JSON integer such as 10**1000 has no float.
        raise ValueError(f"{where} takes a number, not {value!r}") from None
    raw = point.to_wire(number)
    if not math.isfinite(raw):
        raise ValueError(f"{where} takes a finite number, not {value!r}")
    whole = round(raw)
    if not point.in_range(whole) or not _INT32[0] <= whole <= _INT32[1]:
        raise ValueError(f"{where}: {number:g} is outside {self.span(point)}")
    return whole

limits `staticmethod` ¶

limits(point: Point) -> tuple[float | None, float | None]

Return an analog point's range in engineering units, lowest first; None where open.

Source code in src/py1815/master/profile.py

@staticmethod
def limits(point: Point) -> tuple[float | None, float | None]:
    """Return an analog point's range in engineering units, lowest first; None where open."""
    ends = [
        None if bound is None else point.from_wire(float(bound))
        for bound in (point.minimum, point.maximum)
    ]
    if ends[0] is not None and ends[1] is not None and ends[0] > ends[1]:
        ends.reverse()
    return ends[0], ends[1]

span `staticmethod` ¶

span(point: Point) -> str

Return an analog point's range in engineering units, as text.

Source code in src/py1815/master/profile.py

@staticmethod
def span(point: Point) -> str:
    """Return an analog point's range in engineering units, as text."""
    low, high = DerProfile.limits(point)
    units = f" {point.units}" if point.units and point.units.lower() not in ("n/a",) else ""
    lower = "no lower limit" if low is None else f"{low:g}"
    upper = "no upper limit" if high is None else f"{high:g}"
    return f"{lower} to {upper}{units}"

curve_block ¶

curve_block() -> tuple[list[Point], list[Point]]

Return the curve block's outputs and the inputs that read them back, in order.

The selector, the four fields, then X and Y of each point. Raises ValueError when the map does not hold the block or its readback.

Source code in src/py1815/master/profile.py

def curve_block(self) -> tuple[list[Point], list[Point]]:
    """Return the curve block's outputs and the inputs that read them back, in order.

    The selector, the four fields, then X and Y of each point. Raises
    ValueError when the map does not hold the block or its readback.
    """
    outputs: list[Point] = []
    inputs: list[Point] = []
    for offset in range(1 + curves.FIELDS + 2 * curves.MAX_POINTS):
        output = self.map.get(Kind.AO, self._curve_selector + offset)
        if output is None:
            raise ValueError(
                f"the profile has no AO{self._curve_selector + offset}: no curve block "
                f"starts at AO{self._curve_selector}"
            )
        readback = self.mirror(output)
        if readback is None:
            raise ValueError(f"the profile pairs no input with {address(output)}")
        outputs.append(output)
        inputs.append(readback)
    return outputs, inputs

referenced ¶

referenced() -> Point | None

Return the input that says whether a function names the selected curve, if any.

Source code in src/py1815/master/profile.py

def referenced(self) -> Point | None:
    """Return the input that says whether a function names the selected curve, if any."""
    return self.map.get(Kind.BI, CURVE_REFERENCED)

Survey `dataclass` ¶

Which points an outstation reported when asked for everything it serves.

Source code in src/py1815/master/profile.py

@dataclass(frozen=True)
class Survey:
    """Which points an outstation reported when asked for everything it serves."""

    #: Every point reported by any of the reads.
    served: frozenset[tuple[PointType, int]]
    #: The points the class 0 read carried.
    class_0: frozenset[tuple[PointType, int]]
    exchanges: tuple[Exchange, ...]

Der ¶

Bases: _Operations

The DER profile's operations on an outstation over a socket.

Each operation is a coroutine. One profile operation is under way on an outstation at a time: a curve written by one caller cannot have its selector moved by another between the selector and the points.

Source code in src/py1815/master/profile.py

class Der(_Operations):
    """The DER profile's operations on an outstation over a socket.

    Each operation is a coroutine. One profile operation is under way on an
    outstation at a time: a curve written by one caller cannot have its
    selector moved by another between the selector and the points.
    """

    def __init__(self, outstation: Any, profile: PointMap | DerProfile) -> None:
        """
        Args:
            outstation: A :class:`~py1815.master.api.Outstation`.
            profile: The point map it serves, or a :class:`DerProfile` made from one.
        """
        super().__init__(outstation, profile)

    async def run(self, plan: Plan[Any]) -> Any:
        lock = _LOCKS.get(self.target)
        if lock is None:
            lock = _LOCKS[self.target] = asyncio.Lock()
        async with lock:
            return await run_async(self.target, plan)

LoopbackDer ¶

Bases: _Operations

The DER profile's operations on a :class:~py1815.master.loopback.Loopback, at once.

Source code in src/py1815/master/profile.py

class LoopbackDer(_Operations):
    """The DER profile's operations on a :class:`~py1815.master.loopback.Loopback`, at once."""

    def run(self, plan: Plan[Any]) -> Any:
        return run(self.target, plan)

address ¶

address(point: Point) -> str

Return a point's address as text: its kind and index, as in AO87.

Source code in src/py1815/master/profile.py

def address(point: Point) -> str:
    """Return a point's address as text: its kind and index, as in ``AO87``."""
    return f"{point.kind.value}{point.index}"

label ¶

label(point: Point) -> str

Return a point's name without the description the tables append to some.

Source code in src/py1815/master/profile.py

def label(point: Point) -> str:
    """Return a point's name without the description the tables append to some."""
    return _head(point.name)

quality ¶

quality(flags: int | None, reported: bool = True) -> str

Return what a flag octet says of a value: good, offline, comm_lost, restart.

not_reported for a point the outstation did not report, and no_flags for a variation that carries no flag octet, which says nothing either way.

Source code in src/py1815/master/profile.py

def quality(flags: int | None, reported: bool = True) -> str:
    """Return what a flag octet says of a value: ``good``, ``offline``, ``comm_lost``, ``restart``.

    ``not_reported`` for a point the outstation did not report, and
    ``no_flags`` for a variation that carries no flag octet, which says
    nothing either way.
    """
    if not reported:
        return "not_reported"
    if flags is None:
        return "no_flags"
    if flags & AnalogQuality.ONLINE:
        return "good"
    if flags & AnalogQuality.COMM_LOST:
        return "comm_lost"
    if flags & AnalogQuality.RESTART:
        return "restart"
    return "offline"

read_plan ¶

read_plan(points: Sequence[Point]) -> Plan[Readings]

Read points of the profile, and report each in engineering units.

One request, unless the outstation refuses it whole for a range that holds no point it serves: then each range is read again on its own (D93).

Source code in src/py1815/master/profile.py

def read_plan(points: Sequence[Point]) -> Plan[Readings]:
    """Read points of the profile, and report each in engineering units.

    One request, unless the outstation refuses it whole for a range that holds
    no point it serves: then each range is read again on its own (D93).
    """
    if not points:
        raise ValueError("a read names at least one point")
    wanted: dict[PointType, list[int]] = {}
    for point in points:
        wanted.setdefault(POINT_TYPES[point.kind], []).append(point.index)

    def steps() -> Plan[Readings]:
        received, exchanges = yield from _read_points(wanted)
        readings = []
        for point in points:
            decoded = received.get((POINT_TYPES[point.kind], point.index))
            if decoded is None:
                readings.append(Reading(point, None, None, None, reported=False))
            else:
                readings.append(
                    Reading(point, decoded.value, decoded.flags, decoded.time_ms, reported=True)
                )
        return Readings(tuple(readings), exchanges)

    return steps()

carry_out ¶

carry_out(plan: Plan) -> Step

Return the step that carries out a control plan in one turn.

The carrier's own method is used, so nothing is sent between a select and its operate.

Source code in src/py1815/master/profile.py

def carry_out(plan: controls.Plan) -> Step:
    """Return the step that carries out a control plan in one turn.

    The carrier's own method is used, so nothing is sent between a select and
    its operate.
    """
    return lambda target: target._carry_out(plan)  # pylint: disable=protected-access

write_plan ¶

write_plan(
    profile: DerProfile,
    values: Mapping[str, Any],
    *,
    verify: bool = False,
    mode: Mode | str = Mode.DIRECT,
    variation: int | None = None,
) -> Plan[Written]

Write outputs by name in engineering units, in one request, in the order given.

Everything that can be wrong with the write is refused here, before any request is made. variation 3 or 4 sends an analog value as a float in engineering units, which the profile has an outstation take unscaled; otherwise it travels as the whole transmitted number.

Source code in src/py1815/master/profile.py

def write_plan(
    profile: DerProfile,
    values: Mapping[str, Any],
    *,
    verify: bool = False,
    mode: Mode | str = Mode.DIRECT,
    variation: int | None = None,
) -> Plan[Written]:
    """Write outputs by name in engineering units, in one request, in the order given.

    Everything that can be wrong with the write is refused here, before any
    request is made. ``variation`` 3 or 4 sends an analog value as a float in
    engineering units, which the profile has an outstation take unscaled;
    otherwise it travels as the whole transmitted number.
    """
    if not values:
        raise ValueError("a write names at least one output")
    if variation is not None and variation not in (1, 2, 3, 4):
        raise ValueError(f"{variation!r} is not an analog output variation: 1, 2, 3 or 4")
    planned: list[tuple[Point, bool | float, bool | int | float]] = []
    for name, value in values.items():
        point = profile.point(str(name), outputs=True)
        transmitted = profile.transmitted(point, value)
        if isinstance(transmitted, bool):
            planned.append((point, transmitted, transmitted))
        elif variation in (3, 4):
            planned.append((point, float(value), float(value)))
        else:
            planned.append((point, float(value), transmitted))
    commands = [_command(point, sent, variation) for point, _requested, sent in planned]
    plan = controls.Plan(commands, mode)
    mirrors = [profile.mirror(point) for point, _requested, _sent in planned]

    def steps() -> Plan[Written]:
        operated: Operated = yield carry_out(plan)
        verification: Readings | None = None
        if verify:
            wanted = [mirror for mirror in mirrors if mirror is not None]
            if wanted:
                verification = yield from read_plan(wanted)
        setpoints = []
        for (point, requested, sent), status, mirror in zip(
            planned, operated.statuses, mirrors, strict=True
        ):
            readback = None
            if verification is not None and mirror is not None:
                readback = next(r for r in verification.readings if r.point is mirror)
            setpoints.append(Setpoint(point, requested, sent, status, readback))
        return Written(operated, tuple(setpoints), verification)

    return steps()

switch_plan ¶

switch_plan(
    profile: DerProfile,
    name: str,
    enable: bool,
    *,
    mode: Mode | str = Mode.DIRECT,
) -> Plan[Switched]

Enable or disable a function, then read the input that reports whether it is enabled.

Source code in src/py1815/master/profile.py

def switch_plan(
    profile: DerProfile, name: str, enable: bool, *, mode: Mode | str = Mode.DIRECT
) -> Plan[Switched]:
    """Enable or disable a function, then read the input that reports whether it is enabled."""
    function = profile.function(name)
    plan = controls.Plan([controls.binary(function.enable.index, enable)], mode)

    def steps() -> Plan[Switched]:
        operated: Operated = yield carry_out(plan)
        readback: Readings | None = None
        status: Reading | None = None
        if function.status is not None:
            readback = yield from read_plan([function.status])
            status = readback.readings[0]
        return Switched(function, enable, operated, status, readback)

    return steps()

functions_plan ¶

functions_plan(profile: DerProfile) -> Plan[Functions]

Read which functions the outstation supports, and which are enabled, in one request.

Source code in src/py1815/master/profile.py

def functions_plan(profile: DerProfile) -> Plan[Functions]:
    """Read which functions the outstation supports, and which are enabled, in one request."""
    if not profile.functions:
        raise ValueError("the profile has no functions")
    wanted: list[Point] = []
    for function in profile.functions:
        wanted.append(function.supports)
        if function.status is not None:
            wanted.append(function.status)

    def steps() -> Plan[Functions]:
        readings: Readings = yield from read_plan(wanted)
        by_point = {reading.point.address: reading for reading in readings.readings}

        def said(point: Point | None) -> bool | None:
            if point is None:
                return None
            reading = by_point[point.address]
            return bool(reading.raw) if reading.reported else None

        states = tuple(
            FunctionState(function, said(function.supports), said(function.status))
            for function in profile.functions
        )
        return Functions(states, readings)

    return steps()

curve_plan ¶

curve_plan(
    profile: DerProfile,
    number: int | None = None,
    *,
    mode: Mode | str = Mode.DIRECT,
) -> Plan[tuple[Operated | None, Curve]]

Read the curve the curve block shows, selecting curve number first when given.

Selecting a curve writes the selector, so it is a control.

Source code in src/py1815/master/profile.py

def curve_plan(
    profile: DerProfile, number: int | None = None, *, mode: Mode | str = Mode.DIRECT
) -> Plan[tuple[Operated | None, Curve]]:
    """Read the curve the curve block shows, selecting curve ``number`` first when given.

    Selecting a curve writes the selector, so it is a control.
    """
    outputs, inputs = profile.curve_block()
    selection: controls.Plan | None = None
    if number is not None:
        chosen = profile.transmitted(outputs[0], _whole(number, "the curve number"))
        selection = controls.Plan([_command(outputs[0], chosen, None)], mode)

    def steps() -> Plan[tuple[Operated | None, Curve]]:
        selected: Operated | None = None
        if selection is not None:
            selected = yield carry_out(selection)
        curve = yield from _read_curve(profile, inputs)
        return selected, curve

    return steps()

write_curve_plan ¶

write_curve_plan(
    profile: DerProfile,
    number: Any,
    *,
    type: Any,
    x_units: Any,
    y_units: Any,
    points: Sequence[Sequence[Any]],
    mode: Mode | str = Mode.DIRECT,
) -> Plan[CurveWritten]

Write a curve in the order IEEE 1815.2 clause 6.1.3 lays the block out, then read it back.

Three writes: the selector, then the type, the number of points and the units of X and Y, then X and Y of each point. Each is made only when the one before it was accepted, since a field or a point written after a selector the outstation refused would change whichever curve it was already showing. The curve is read back whatever happened.

Source code in src/py1815/master/profile.py

def write_curve_plan(
    profile: DerProfile,
    number: Any,
    *,
    type: Any,  # pylint: disable=redefined-builtin
    x_units: Any,
    y_units: Any,
    points: Sequence[Sequence[Any]],
    mode: Mode | str = Mode.DIRECT,
) -> Plan[CurveWritten]:
    """Write a curve in the order IEEE 1815.2 clause 6.1.3 lays the block out, then read it back.

    Three writes: the selector, then the type, the number of points and the
    units of X and Y, then X and Y of each point. Each is made only when the
    one before it was accepted, since a field or a point written after a
    selector the outstation refused would change whichever curve it was
    already showing. The curve is read back whatever happened.
    """
    outputs, inputs = profile.curve_block()
    chosen = _whole(number, "the curve number")
    fields = [
        _whole(type, "the curve type"),
        len(points),
        _whole(x_units, "the units of X"),
        _whole(y_units, "the units of Y"),
    ]
    if len(points) > curves.MAX_POINTS:
        raise ValueError(f"a curve has at most {curves.MAX_POINTS} points, not {len(points)}")
    pairs: list[tuple[int, int]] = []
    for position, pair in enumerate(points, start=1):
        if not isinstance(pair, Sequence) or isinstance(pair, (str, bytes)) or len(pair) != 2:
            raise ValueError(f"point {position} of the curve is an X and a Y")
        pairs.append(
            (_whole(pair[0], f"X of point {position}"), _whole(pair[1], f"Y of point {position}"))
        )
    flat = [value for pair in pairs for value in pair]
    writes = [
        [(outputs[0], chosen)],
        list(zip(outputs[1 : 1 + curves.FIELDS], fields, strict=True)),
        list(zip(outputs[1 + curves.FIELDS :], flat, strict=False)),
    ]
    plans = [
        controls.Plan(
            [_command(point, profile.transmitted(point, value), None) for point, value in step],
            mode,
        )
        for step in writes
        if step
    ]

    def steps() -> Plan[CurveWritten]:
        done: list[Operated] = []
        for plan in plans:
            operated: Operated = yield carry_out(plan)
            done.append(operated)
            if operated.accepted is not True:
                break
        readback = yield from _read_curve(profile, inputs)
        return CurveWritten(
            number=chosen,
            type=fields[0],
            x_units=fields[2],
            y_units=fields[3],
            points=tuple(pairs),
            steps=tuple(done),
            readback=readback,
        )

    return steps()

read_device_profile ¶

read_device_profile(text: str) -> tuple[Declared, ...]

Return the points a DNP3 Device Profile document declares.

Reads the point lists of the form :mod:py1815.profile.device_profile writes, schema version 2.12.00. A counter that declares a frozen counter beside it is two points. A document that declares a document type is refused, since nothing a Device Profile holds needs one and an entity declared there is how an XML document is made to expand without bound.

Source code in src/py1815/master/profile.py

def read_device_profile(text: str) -> tuple[Declared, ...]:
    """Return the points a DNP3 Device Profile document declares.

    Reads the point lists of the form :mod:`py1815.profile.device_profile`
    writes, schema version 2.12.00. A counter that declares a frozen counter
    beside it is two points. A document that declares a document type is
    refused, since nothing a Device Profile holds needs one and an entity
    declared there is how an XML document is made to expand without bound.
    """
    if "<!DOCTYPE" in text.upper():
        raise ValueError("a Device Profile document has no document type declaration")
    try:
        root = ElementTree.fromstring(text)
    except ElementTree.ParseError as error:
        raise ValueError(f"the Device Profile document is not XML: {error}") from None
    lists = next(
        (element for element in root.iter() if _local(element.tag) == "dataPointsList"), None
    )
    if lists is None:
        raise ValueError("the document has no dataPointsList, so it declares no points")
    declared: list[Declared] = []
    for section in lists:
        found = _LISTS.get(_local(section.tag))
        if found is None:
            continue
        element_name, point_type = found
        points = _child(section, "dataPoints")
        for element in points if points is not None else ():
            if _local(element.tag) != element_name:
                continue
            index_text = _text(element, "index")
            if index_text is None or not index_text.isdigit():
                raise ValueError(f"a {element_name} in the document has no index")
            index = int(index_text)
            name = _text(element, "name")
            if point_type is PointType.COUNTER:
                declared.append(
                    Declared(
                        point_type,
                        index,
                        name,
                        _class(_text(element, "counterEventClass")),
                        _included(_text(element, "countersIncludedInClass0")),
                        None,
                    )
                )
                if (_text(element, "frozenCounterExists") or "").casefold() == "true":
                    declared.append(
                        Declared(
                            PointType.FROZEN_COUNTER,
                            index,
                            name,
                            _class(_text(element, "frozenCounterEventClass")),
                            _included(_text(element, "frozenCountersIncludedInClass0")),
                            None,
                        )
                    )
                continue
            deadband = None
            data = _child(element, "dnpData")
            if data is not None and _text(data, "deadband") is not None:
                try:
                    deadband = float(_text(data, "deadband") or "")
                except ValueError:
                    deadband = None
                # NaN and infinity are not deadbands, and are not valid JSON.
                if deadband is not None and not math.isfinite(deadband):
                    deadband = None
            declared.append(
                Declared(
                    point_type,
                    index,
                    name,
                    _class(_text(element, "changeEventClass")),
                    _included(_text(element, "includedInClass0Response")),
                    deadband,
                )
            )
    return tuple(declared)

survey_plan ¶

survey_plan(
    wanted: Iterable[tuple[PointType, int]],
) -> Plan[Survey]

Read which points an outstation serves.

A class 0 read, then a read of output status, then a read by range of every point in wanted that neither returned. An outstation that does not answer the class 0 read is asked nothing more.

Source code in src/py1815/master/profile.py

def survey_plan(wanted: Iterable[tuple[PointType, int]]) -> Plan[Survey]:
    """Read which points an outstation serves.

    A class 0 read, then a read of output status, then a read by range of
    every point in ``wanted`` that neither returned. An outstation that does
    not answer the class 0 read is asked nothing more.
    """
    wanted = tuple(wanted)

    def steps() -> Plan[Survey]:
        class_0: Exchange = yield lambda target: target.scan("class0")
        if not class_0.fragments:
            return Survey(frozenset(), frozenset(), (class_0,))
        outputs: Exchange = yield lambda target: target.scan("outputs")
        carried = set(_static(class_0.objects))
        seen = carried | set(_static(outputs.objects))
        exchanges = [class_0, outputs]
        missing: dict[PointType, list[int]] = {}
        for key in wanted:
            if key not in seen:
                missing.setdefault(key[0], []).append(key[1])
        if missing:
            found, looked = yield from _read_points(missing)
            exchanges += looked
            seen |= set(found)
        return Survey(frozenset(seen), frozenset(carried), tuple(exchanges))

    return steps()

compare_plan ¶

compare_plan(document: str) -> Plan[Comparison]

Compare what an outstation serves with what its Device Profile document declares.

A class 0 read, then a read of output status, then a read of every declared point neither returned, by range. A point no read returned is absent. Whether each served point is in class 0 is checked against the class 0 read. A point's event class and deadband cannot be read from an outstation, so they are reported as declared.

Source code in src/py1815/master/profile.py

def compare_plan(document: str) -> Plan[Comparison]:
    """Compare what an outstation serves with what its Device Profile document declares.

    A class 0 read, then a read of output status, then a read of every
    declared point neither returned, by range. A point no read returned is
    absent. Whether each served point is in class 0 is checked against the
    class 0 read. A point's event class and deadband cannot be read from an
    outstation, so they are reported as declared.
    """
    declared = read_device_profile(document)
    by_point = {(entry.type, entry.index): entry for entry in declared}

    def steps() -> Plan[Comparison]:
        survey = yield from survey_plan(by_point)
        carried, seen, exchanges = survey.class_0, survey.served, survey.exchanges
        absent = tuple(entry for key, entry in by_point.items() if key not in seen)
        undeclared = tuple(sorted((key for key in seen if key not in by_point), key=_order))
        served = tuple(entry for key, entry in by_point.items() if key in seen)
        misplaced = tuple(
            (entry, (entry.type, entry.index) in carried)
            for entry in served
            if entry.class_0 is not None and entry.class_0 != ((entry.type, entry.index) in carried)
        )
        return Comparison(declared, absent, undeclared, misplaced, served, exchanges)

    return steps()

run ¶

run(target: Any, plan: Plan[T]) -> T

Carry out a plan on a master whose requests return at once, as a Loopback's do.

Source code in src/py1815/master/profile.py

def run(target: Any, plan: Plan[T]) -> T:
    """Carry out a plan on a master whose requests return at once, as a Loopback's do."""
    try:
        step = next(plan)
        while True:
            step = plan.send(step(target))
    except StopIteration as done:
        return done.value  # type: ignore[no-any-return]

run_async `async` ¶

run_async(target: Any, plan: Plan[T]) -> T

Carry out a plan on a master whose requests are awaited, as an Outstation's are.

Source code in src/py1815/master/profile.py

async def run_async(target: Any, plan: Plan[T]) -> T:
    """Carry out a plan on a master whose requests are awaited, as an Outstation's are."""
    try:
        step = next(plan)
        while True:
            outcome: Awaitable[Any] = step(target)
            step = plan.send(await outcome)
    except StopIteration as done:
        return done.value  # type: ignore[no-any-return]

Evaluating an outstation¶

Evaluating an outstation is the guide.

py1815.master.evaluate ¶

Run checks against an outstation, and report how each ended.

What py1815-master evaluate does: connect to an outstation, carry out the checks of :mod:py1815.master.checks, and print a report with a verdict for each. A check that writes to the outstation is run only when the run is allowed to command, and a check that has the master break a protocol rule only when the run is allowed to deviate. The report can also be written as JSON, and the traffic as a pcap file in which each check's frames can be found by number.

From Python:

.. code-block:: python

report = await evaluate(outstation, point_map, commanding=True)
for line in report.lines():
    print(line)

:func:evaluate takes a connected :class:~py1815.master.api.Outstation. :func:evaluate_loopback runs the same checks against a session in this process, with no socket and no waiting.

Report `dataclass` ¶

How every check of a run ended.

Source code in src/py1815/master/evaluate.py

@dataclass(frozen=True)
class Report:
    """How every check of a run ended."""

    #: What was evaluated, as the caller named it.
    outstation: str
    #: When the run began, in seconds since the epoch.
    started: float
    #: Whether the run was allowed to write to the outstation.
    commanding: bool
    #: The edition of the profile tables the checks were read from.
    edition: str
    results: tuple[Result, ...]
    #: Whether the run was allowed to have the master break protocol rules.
    deviating: bool = False

    def count(self, verdict: Verdict) -> int:
        """Return how many checks ended with a verdict."""
        return sum(1 for result in self.results if result.verdict is verdict)

    @property
    def failed(self) -> bool:
        """Whether any check failed."""
        return self.count(Verdict.FAILED) > 0

    def describe(self) -> dict[str, Any]:
        """Return the report as a JSON-compatible dict."""
        try:
            version = importlib.metadata.version("py1815")
        except importlib.metadata.PackageNotFoundError:
            version = "unknown"
        began = datetime.datetime.fromtimestamp(self.started, datetime.UTC)
        return {
            "tool": "py1815-master evaluate",
            "version": version,
            "outstation": self.outstation,
            "started": began.isoformat(timespec="seconds"),
            "allow_control": self.commanding,
            "allow_deviations": self.deviating,
            "tables": self.edition,
            "summary": {verdict.value: self.count(verdict) for verdict in Verdict},
            "results": [result.describe() for result in self.results],
        }

    def render(self) -> str:
        """Return the report as formatted JSON."""
        return json.dumps(self.describe(), indent=2) + "\n"

    def lines(self) -> list[str]:
        """Return the report as lines of text: a line for each check, then the totals."""
        began = datetime.datetime.fromtimestamp(self.started, datetime.UTC)
        writes = "allowed to write" if self.commanding else "read only"
        if self.deviating:
            writes += ", allowed to deviate"
        lines = [
            f"{self.outstation}, {began.isoformat(timespec='seconds')}, {writes}",
            f"profile tables: {self.edition or 'edition not stated'}",
            "",
        ]
        width = max((len(result.id) for result in self.results), default=0)
        for result in self.results:
            lines.append(f"{result.id:<{width}}  {_WORDS[result.verdict]:<14}  {result.title}")
            indent = " " * (width + 2)
            if result.detail:
                lines.append(f"{indent}{result.detail}")
            lines.extend(f"{indent}note: {note}" for note in result.notes)
            if result.verdict is Verdict.FAILED and result.packets is not None:
                first, last = result.packets
                lines.append(f"{indent}packets {first} to {last} of the capture")
            elif result.verdict is Verdict.FAILED and result.frames is not None:
                lines.append(f"{indent}frames {result.frames[0]} to {result.frames[1]}")
        totals = ", ".join(
            f"{self.count(verdict)} {_WORDS[verdict].lower()}"
            for verdict in Verdict
            if self.count(verdict)
        )
        lines += ["", f"{len(self.results)} checks: {totals or 'none'}"]
        return lines

failed `property` ¶

failed: bool

Whether any check failed.

count ¶

count(verdict: Verdict) -> int

Return how many checks ended with a verdict.

Source code in src/py1815/master/evaluate.py

def count(self, verdict: Verdict) -> int:
    """Return how many checks ended with a verdict."""
    return sum(1 for result in self.results if result.verdict is verdict)

describe ¶

describe() -> dict[str, Any]

Return the report as a JSON-compatible dict.

Source code in src/py1815/master/evaluate.py

def describe(self) -> dict[str, Any]:
    """Return the report as a JSON-compatible dict."""
    try:
        version = importlib.metadata.version("py1815")
    except importlib.metadata.PackageNotFoundError:
        version = "unknown"
    began = datetime.datetime.fromtimestamp(self.started, datetime.UTC)
    return {
        "tool": "py1815-master evaluate",
        "version": version,
        "outstation": self.outstation,
        "started": began.isoformat(timespec="seconds"),
        "allow_control": self.commanding,
        "allow_deviations": self.deviating,
        "tables": self.edition,
        "summary": {verdict.value: self.count(verdict) for verdict in Verdict},
        "results": [result.describe() for result in self.results],
    }

render ¶

render() -> str

Return the report as formatted JSON.

Source code in src/py1815/master/evaluate.py

def render(self) -> str:
    """Return the report as formatted JSON."""
    return json.dumps(self.describe(), indent=2) + "\n"

lines ¶

lines() -> list[str]

Return the report as lines of text: a line for each check, then the totals.

Source code in src/py1815/master/evaluate.py

def lines(self) -> list[str]:
    """Return the report as lines of text: a line for each check, then the totals."""
    began = datetime.datetime.fromtimestamp(self.started, datetime.UTC)
    writes = "allowed to write" if self.commanding else "read only"
    if self.deviating:
        writes += ", allowed to deviate"
    lines = [
        f"{self.outstation}, {began.isoformat(timespec='seconds')}, {writes}",
        f"profile tables: {self.edition or 'edition not stated'}",
        "",
    ]
    width = max((len(result.id) for result in self.results), default=0)
    for result in self.results:
        lines.append(f"{result.id:<{width}}  {_WORDS[result.verdict]:<14}  {result.title}")
        indent = " " * (width + 2)
        if result.detail:
            lines.append(f"{indent}{result.detail}")
        lines.extend(f"{indent}note: {note}" for note in result.notes)
        if result.verdict is Verdict.FAILED and result.packets is not None:
            first, last = result.packets
            lines.append(f"{indent}packets {first} to {last} of the capture")
        elif result.verdict is Verdict.FAILED and result.frames is not None:
            lines.append(f"{indent}frames {result.frames[0]} to {result.frames[1]}")
    totals = ", ".join(
        f"{self.count(verdict)} {_WORDS[verdict].lower()}"
        for verdict in Verdict
        if self.count(verdict)
    )
    lines += ["", f"{len(self.results)} checks: {totals or 'none'}"]
    return lines

evaluate `async` ¶

evaluate(
    outstation: Any,
    point_map: PointMap | DerProfile,
    *,
    checks: Sequence[Check] | None = None,
    commanding: bool = False,
    deviating: bool = False,
    settle: float = DEFAULT_SETTLE,
    curves: int | None = None,
    device_profile: str | None = None,
    capture: Any = None,
    pause: Callable[
        [float], Awaitable[None]
    ] = asyncio.sleep,
    on_result: Callable[[Result], None] | None = None,
) -> Report

Run checks against a connected outstation, and return the report.

Parameters:

Name	Type	Description	Default
`outstation`	`Any`	A connected :class:`~py1815.master.api.Outstation`.	required
`point_map`	`PointMap \| DerProfile`	The profile's points, as the outstation is expected to serve them.	required
`checks`	`Sequence[Check] \| None`	The checks to run, in order. Defaults to the whole catalog.	`None`
`commanding`	`bool`	Run the checks that write to the outstation. They change its settings, enable and disable its functions, and stop and start it.	`False`
`deviating`	`bool`	Run the checks that have the master break a protocol rule on purpose. They send the outstation frames it has to discard and withhold confirmations it asked for.	`False`
`settle`	`float`	Seconds to wait for the outstation to reach a state it was commanded to.	`DEFAULT_SETTLE`
`curves`	`int \| None`	How many curves the outstation stores. Found by selecting each in turn when not given.	`None`
`device_profile`	`str \| None`	The text of the outstation's Device Profile document.	`None`
`capture`	`Any`	The :class:`~py1815.master.capture.Capture` the outstation's frames are being written to. Each result then gives the numbers of its packets in it.	`None`
`pause`	`Callable[[float], Awaitable[None]]`	Awaited with the seconds of each wait.	`sleep`
`on_result`	`Callable[[Result], None] \| None`	Called with each result as its check ends.	`None`

Raises :class:~py1815.master.bench.NoAnswer when the outstation does not answer the first read.

Source code in src/py1815/master/evaluate.py

async def evaluate(
    outstation: Any,
    point_map: PointMap | DerProfile,
    *,
    checks: Sequence[Check] | None = None,
    commanding: bool = False,
    deviating: bool = False,
    settle: float = DEFAULT_SETTLE,
    curves: int | None = None,
    device_profile: str | None = None,
    capture: Any = None,
    pause: Callable[[float], Awaitable[None]] = asyncio.sleep,
    on_result: Callable[[Result], None] | None = None,
) -> Report:
    """Run checks against a connected outstation, and return the report.

    Args:
        outstation: A connected :class:`~py1815.master.api.Outstation`.
        point_map: The profile's points, as the outstation is expected to serve them.
        checks: The checks to run, in order. Defaults to the whole catalog.
        commanding: Run the checks that write to the outstation. They change
            its settings, enable and disable its functions, and stop and
            start it.
        deviating: Run the checks that have the master break a protocol rule
            on purpose. They send the outstation frames it has to discard and
            withhold confirmations it asked for.
        settle: Seconds to wait for the outstation to reach a state it was
            commanded to.
        curves: How many curves the outstation stores. Found by selecting
            each in turn when not given.
        device_profile: The text of the outstation's Device Profile document.
        capture: The :class:`~py1815.master.capture.Capture` the outstation's
            frames are being written to. Each result then gives the numbers of
            its packets in it.
        pause: Awaited with the seconds of each wait.
        on_result: Called with each result as its check ends.

    Raises :class:`~py1815.master.bench.NoAnswer` when the outstation does
    not answer the first read.
    """
    bench = _bench(point_map, settle, curves, device_profile, capture)
    started = time.time()
    plan = benches.run_plan(
        bench,
        CATALOG if checks is None else checks,
        commanding=commanding,
        deviating=deviating,
        on_result=on_result,
    )
    try:
        results = await benches.drive_async(outstation, plan, pause)
    finally:
        # A run that is cancelled or fails part way through a check never
        # reaches the end of that check. Its deviations are turned off and
        # the master is let go here, which sends nothing.
        bench.restore(outstation)
    name = getattr(outstation, "name", "outstation")
    return Report(str(name), started, commanding, bench.map.edition, results, deviating)

evaluate_loopback ¶

evaluate_loopback(
    loopback: Any,
    point_map: PointMap | DerProfile,
    *,
    checks: Sequence[Check] | None = None,
    commanding: bool = False,
    deviating: bool = False,
    settle: float = DEFAULT_SETTLE,
    curves: int | None = None,
    device_profile: str | None = None,
    advance: Callable[[float], None] | None = None,
) -> Report

Run checks against a :class:~py1815.master.loopback.Loopback, and return the report.

advance is called with the seconds of each wait, so the caller can move a simulated device forward. Nothing waits in real time.

Source code in src/py1815/master/evaluate.py

def evaluate_loopback(
    loopback: Any,
    point_map: PointMap | DerProfile,
    *,
    checks: Sequence[Check] | None = None,
    commanding: bool = False,
    deviating: bool = False,
    settle: float = DEFAULT_SETTLE,
    curves: int | None = None,
    device_profile: str | None = None,
    advance: Callable[[float], None] | None = None,
) -> Report:
    """Run checks against a :class:`~py1815.master.loopback.Loopback`, and return the report.

    ``advance`` is called with the seconds of each wait, so the caller can
    move a simulated device forward. Nothing waits in real time.
    """
    bench = _bench(point_map, settle, curves, device_profile)
    started = time.time()
    plan = benches.run_plan(
        bench, CATALOG if checks is None else checks, commanding=commanding, deviating=deviating
    )
    try:
        results = benches.drive(loopback, plan, advance or (lambda seconds: None))
    finally:
        bench.restore(loopback)
    return Report("loopback", started, commanding, bench.map.edition, results, deviating)

catalog_lines ¶

catalog_lines() -> list[str]

Return the catalog as lines of text: each check's identifier, set and title.

Source code in src/py1815/master/evaluate.py

def catalog_lines() -> list[str]:
    """Return the catalog as lines of text: each check's identifier, set and title."""
    width = max(len(check.id) for check in CATALOG)
    lines = []
    for check in CATALOG:
        kind = "writes" if check.commands else "deviates" if check.deviates else "reads"
        lines.append(f"{check.id:<{width}}  {check.group:<14}  {kind:<8}  {check.title}")
    return lines

run ¶

run(args: Namespace, config: MasterConfig) -> int

Evaluate an outstation as args and config say. Return the exit status.

0 when no check failed, 1 when one did, and 2 when the run could not be made: a setting that cannot be used, no tables, or no answer.

Source code in src/py1815/master/evaluate.py

def run(args: argparse.Namespace, config: MasterConfig) -> int:
    """Evaluate an outstation as ``args`` and ``config`` say. Return the exit status.

    0 when no check failed, 1 when one did, and 2 when the run could not be
    made: a setting that cannot be used, no tables, or no answer.
    """
    if args.list:
        for line in catalog_lines():
            print(line)
        return 0
    try:
        chosen = _chosen(config, args.name)
        checks = select(config.evaluate.checks)
        point_map = load.load(config.tables, Composition())
        device_profile = None
        if chosen.device_profile is not None:
            device_profile = pathlib.Path(chosen.device_profile).read_text(encoding="utf-8")
    except (MapError, ValueError, OSError) as error:
        print(str(error), file=sys.stderr)
        return 2
    try:
        report = asyncio.run(_run(config, chosen, point_map, checks, device_profile))
    except NoAnswer as error:
        print(f"{chosen.name} at {chosen.host}:{chosen.port}: {error}", file=sys.stderr)
        return 2
    except (OSError, ValueError) as error:
        print(
            f"cannot evaluate {chosen.name} at {chosen.host}:{chosen.port}: {error}",
            file=sys.stderr,
        )
        return 2
    for line in report.lines():
        print(line)
    if config.evaluate.report is not None:
        try:
            pathlib.Path(config.evaluate.report).write_text(
                report.render(), encoding="utf-8", newline="\n"
            )
        except OSError as error:
            print(f"cannot write the report: {error}", file=sys.stderr)
            return 2
        print(f"wrote {config.evaluate.report}")
    return 1 if report.failed else 0

py1815.master.bench ¶

Checks of an outstation: what one is, how it is run, and what it reports.

A check is a short named procedure with a verdict. It is written once, as a plan that yields one request at a time, so the same check runs against an outstation on a socket and against a session in the same process. A check fails by raising :class:Failed with what the outstation did, and is not applicable when the outstation does not serve what it needs.

:class:Bench is what a check works with: the profile's point map, the points the outstation serves, and the requests a check is made of. Each request that is not answered fails the check. Nothing here knows any particular check; the catalog is :mod:py1815.master.checks.

Verdict ¶

Bases: Enum

How a check ended.

Source code in src/py1815/master/bench.py

class Verdict(Enum):
    """How a check ended."""

    #: The outstation did everything the check requires.
    PASSED = "passed"
    #: The outstation did something the check does not allow.
    FAILED = "failed"
    #: The outstation does not serve what the check needs.
    NOT_APPLICABLE = "not_applicable"
    #: The check was not carried out.
    NOT_RUN = "not_run"

Failed ¶

Bases: Exception

The outstation did something a check does not allow. The message says what.

Source code in src/py1815/master/bench.py

class Failed(Exception):
    """The outstation did something a check does not allow. The message says what."""

Unanswered ¶

Bases: Failed

The outstation did not answer a request. Waiting and reading again will not help.

Source code in src/py1815/master/bench.py

class Unanswered(Failed):
    """The outstation did not answer a request. Waiting and reading again will not help."""

NotApplicable ¶

Bases: Exception

The outstation does not serve what a check needs. The message says what.

Source code in src/py1815/master/bench.py

class NotApplicable(Exception):
    """The outstation does not serve what a check needs. The message says what."""

NoAnswer ¶

Bases: RuntimeError

The outstation answered nothing at all, so no check could be run.

Source code in src/py1815/master/bench.py

class NoAnswer(RuntimeError):
    """The outstation answered nothing at all, so no check could be run."""

Pause `dataclass` ¶

A step of a plan that waits and sends nothing.

Source code in src/py1815/master/bench.py

@dataclass(frozen=True)
class Pause:
    """A step of a plan that waits and sends nothing."""

    seconds: float

    def __call__(self, target: Any) -> None:
        """Do nothing: a driver that knows no waits carries on at once."""

Check `dataclass` ¶

One named procedure.

Source code in src/py1815/master/bench.py

@dataclass(frozen=True)
class Check:
    """One named procedure."""

    #: A short identifier, as in ``MON-001``.
    id: str
    title: str
    #: Whether the check writes to the outstation.
    commands: bool
    procedure: Callable[[Bench], Plan[None]]
    #: The set the check belongs to.
    group: str = "der"
    #: Whether the check has the master break a protocol rule on purpose.
    deviates: bool = False

Result `dataclass` ¶

How one check ended.

Source code in src/py1815/master/bench.py

@dataclass(frozen=True)
class Result:
    """How one check ended."""

    id: str
    title: str
    verdict: Verdict
    #: Why the check failed, was not applicable or was not run. Empty when it passed.
    detail: str = ""
    #: What the check saw that a reader of the report should know.
    notes: tuple[str, ...] = ()
    #: Requests the check sent.
    requests: int = 0
    seconds: float = 0.0
    #: The ids of the first and last frame of the check in the master's
    #: trace, which counts DNP3 frames from 1. None when the check sent
    #: nothing or the master keeps no trace.
    frames: tuple[int, int] | None = None
    #: The numbers of the first and last packet of the check in the capture
    #: file, counted from 1 as Wireshark counts them. A capture also holds
    #: each connection's handshake, so these are not the frame ids. None when
    #: no capture is written or the check sent nothing.
    packets: tuple[int, int] | None = None

    def describe(self) -> dict[str, Any]:
        """Return the result as a JSON-compatible dict."""
        return {
            "id": self.id,
            "title": self.title,
            "verdict": self.verdict.value,
            "detail": self.detail,
            "notes": list(self.notes),
            "requests": self.requests,
            "seconds": round(self.seconds, 3),
            "frames": None if self.frames is None else list(self.frames),
            "packets": None if self.packets is None else list(self.packets),
        }

describe ¶

describe() -> dict[str, Any]

Return the result as a JSON-compatible dict.

Source code in src/py1815/master/bench.py

def describe(self) -> dict[str, Any]:
    """Return the result as a JSON-compatible dict."""
    return {
        "id": self.id,
        "title": self.title,
        "verdict": self.verdict.value,
        "detail": self.detail,
        "notes": list(self.notes),
        "requests": self.requests,
        "seconds": round(self.seconds, 3),
        "frames": None if self.frames is None else list(self.frames),
        "packets": None if self.packets is None else list(self.packets),
    }

Bench ¶

What a check works with.

Every method that sends a request is a plan: call it with yield from. A request the outstation does not answer raises :class:Unanswered.

Source code in src/py1815/master/bench.py

class Bench:
    """What a check works with.

    Every method that sends a request is a plan: call it with ``yield from``.
    A request the outstation does not answer raises :class:`Unanswered`.
    """

    def __init__(
        self,
        profile: DerProfile,
        *,
        settle: float = DEFAULT_SETTLE,
        curves: int | None = None,
        device_profile: str | None = None,
        capture: Any = None,
    ) -> None:
        """
        Args:
            profile: The point map the outstation is checked against.
            settle: Seconds to wait for the outstation to reach a state it
                was commanded to.
            curves: How many curves the outstation stores, or None to find out
                by selecting each in turn.
            device_profile: The text of the outstation's Device Profile
                document, or None when there is none.
            capture: The :class:`~py1815.master.capture.Capture` the run's
                frames are written to, or None. Each result then gives the
                numbers of its packets in it.
        """
        self.capture = capture
        self.profile = profile
        self.map = profile.map
        self.settle = settle
        self.curves = curves
        self.device_profile = device_profile
        #: The points the outstation reported when the run began.
        self.served: frozenset[Address] = frozenset()
        #: A title for the check being run that says more than the catalog's.
        self.title: str | None = None
        self.requests = 0
        self._notes: list[str] = []
        self._afterwards: list[tuple[str, Callable[[], Plan[Any]]]] = []
        #: The master's deviations as the check found them, once it has changed any.
        self._deviations: dict[str, bool] | None = None
        #: Whether the check began with every deviation turned off.
        self._isolated = False

    # ------------------------------------------------------------ a check's own

    def begin(self) -> None:
        """Forget what the last check noted, counted and left to be put back."""
        self.title = None
        self.requests = 0
        self._notes = []
        self._afterwards = []
        self._deviations = None
        self._isolated = False

    def note(self, text: str) -> None:
        """Record something the report should say about this check."""
        if text not in self._notes:
            self._notes.append(text)

    @property
    def notes(self) -> tuple[str, ...]:
        return tuple(self._notes)

    @staticmethod
    def require(holds: object, otherwise: str) -> None:
        """Fail the check with ``otherwise`` unless ``holds``."""
        if not holds:
            raise Failed(otherwise)

    def afterwards(self, what: str, plan: Callable[[], Plan[Any]]) -> None:
        """Have ``plan`` carried out when the check ends, however it ends.

        A check uses this to put back what it changed. The plans are carried
        out last one first.
        """
        self._afterwards.append((what, plan))

    def put_back(self) -> Plan[None]:
        """Carry out what the check left to be done afterwards."""
        while self._afterwards:
            what, plan = self._afterwards.pop()
            try:
                yield from plan()
            except Failed as failure:
                self.note(f"{what} could not be put back: {failure}")

    # ---------------------------------------------------------------- serving

    def supported(self, kind: Kind, index: int) -> bool:
        """Return whether the outstation serves a point."""
        return (kind, index) in self.served

    def supported_in(self, kind: Kind, indices: Sequence[int]) -> list[Point]:
        """Return the points of one kind the outstation serves among ``indices``."""
        return [self.map.point(kind, i) for i in indices if self.supported(kind, i)]

    def readback(self, output: Point) -> Point | None:
        """Return the input that reads an output back, or None when none is served."""
        mirror = self.profile.mirror(output)
        if mirror is None or mirror.address not in self.served:
            return None
        return mirror

    # ---------------------------------------------------------------- reading

    def _made(self, exchanges: Sequence[Exchange], what: str) -> None:
        self.requests += len(exchanges)
        for exchange in exchanges:
            if not exchange.fragments:
                raise Unanswered(f"the outstation did not answer {what}")

    def read(self, points: Sequence[Point]) -> Plan[Readings]:
        """Read points in one request."""
        readings: Readings = yield from read_plan(points)
        listed = ", ".join(address(point) for point in points[:4])
        more = f" and {len(points) - 4} more" if len(points) > 4 else ""
        self._made(readings.exchanges, f"a read of {listed}{more}")
        return readings

    def obtain(self, point: Point) -> Plan[Reading]:
        """Read one point and require the outstation to report it."""
        readings = yield from self.read([point])
        reading = readings.readings[0]
        self.require(reading.reported, f"{name(point)} could not be read")
        return reading

    def online(self, point: Point) -> Plan[Reading]:
        """Read a point and require it to be ONLINE with no other quality flag.

        A variation that carries no flags says the same thing.
        """
        reading = yield from self.obtain(point)
        flags = _flags(reading)
        self.require(
            flags is None or flags == AnalogQuality.ONLINE,
            f"{name(point)} is reported with flags 0x{flags or 0:02X}, and not as ONLINE alone",
        )
        return reading

    def offline(self, point: Point) -> Plan[Reading]:
        """Read a point and require it to carry a value and no quality flag at all."""
        reading = yield from self.obtain(point)
        flags = _flags(reading)
        said = "no flags, which says ONLINE" if flags is None else f"flags 0x{flags:02X}"
        self.require(flags == 0, f"{name(point)} is reported with {said}, and not as offline")
        return reading

    def state(self, point: Point) -> Plan[bool]:
        """Return a binary point's state, which is required to be ONLINE."""
        reading = yield from self.online(point)
        return bool(reading.raw)

    def value(self, point: Point) -> Plan[float]:
        """Return an analog point's value in engineering units, required to be ONLINE."""
        reading = yield from self.online(point)
        return self._engineering(reading)

    def raw(self, point: Point) -> Plan[float]:
        """Return an analog point's value in engineering units, whatever its flags say."""
        reading = yield from self.obtain(point)
        return self._engineering(reading)

    @staticmethod
    def _engineering(reading: Reading) -> float:
        value = reading.value
        if value is None or isinstance(value, bool):
            raise Failed(f"{name(reading.point)} did not report a number")
        return float(value)

    # ---------------------------------------------------------------- writing

    def _operate(self, commands: Sequence[Command], mode: Mode, what: str) -> Plan[CommandStatus]:
        operated: Operated = yield carry_out(controls.Plan(commands, mode))
        self._made(operated.exchanges, what)
        answered = operated.statuses[0]
        if answered.status is None:
            raise Failed(f"the response to {what} did not carry the control back")
        if answered.status is CommandStatus.SUCCESS and not answered.echoed:
            raise Failed(f"the response to {what} carried back a different control")
        return answered.status

    def set(self, output: Point | int, raw: int) -> Plan[CommandStatus]:
        """Direct operate one analog output with a transmitted value. Return the status."""
        index = output if isinstance(output, int) else output.index
        status = yield from self._operate(
            [controls.analog(index, raw)], Mode.DIRECT, f"a write of {raw} to AO{index}"
        )
        return status

    def latch(self, output: Point | int, on: bool) -> Plan[CommandStatus]:
        """Select and then operate a latch of one binary output. Return the status.

        The status is the select's when the outstation refused the select.
        """
        index = output if isinstance(output, int) else output.index
        status = yield from self._operate(
            [controls.binary(index, on)],
            Mode.SELECT,
            f"a latch {'on' if on else 'off'} of BO{index}",
        )
        return status

    def accepted(self, status: CommandStatus, what: str) -> None:
        """Fail the check unless the outstation accepted a control."""
        self.require(
            status is CommandStatus.SUCCESS, f"{what} was refused with status {status.name}"
        )

    def refused(self, status: CommandStatus, what: str, usual: CommandStatus) -> None:
        """Fail the check if the outstation accepted a control it is to refuse.

        A refusal with a status other than ``usual`` is noted and not failed:
        which status refuses a control is the outstation's to choose.
        """
        self.require(status is not CommandStatus.SUCCESS, f"{what} was accepted")
        if status is not usual:
            self.note(f"{what} was refused with {status.name}, where {usual.name} is usual")

    def write_curve(
        self,
        number: int,
        *,
        type: int,  # pylint: disable=redefined-builtin
        x_units: int,
        y_units: int,
        points: Sequence[Sequence[int]],
    ) -> Plan[CurveWritten]:
        """Write a curve in the profile's three steps and read it back."""
        written: CurveWritten = yield from write_curve_plan(
            self.profile, number, type=type, x_units=x_units, y_units=y_units, points=points
        )
        what = f"the write of curve {number}"
        for step in written.steps:
            self._made(step.exchanges, what)
        for read in written.readback.reads:
            self._made(read.exchanges, f"the read of curve {number}")
        return written

    def compare(self, document: str) -> Plan[Comparison]:
        """Compare what the outstation serves with a Device Profile document."""
        comparison: Comparison = yield from compare_plan(document)
        self._made(comparison.exchanges, "the reads of the points its Device Profile declares")
        return comparison

    # -------------------------------------------------------------- deviating

    def deviate(self, *names: str) -> Plan[None]:
        """Have the master break the named protocol rules until the check ends.

        The master's deviations are put back as the check found them when the
        check ends, or sooner by :meth:`conform`.
        """

        def turn_on(target: Any) -> bool:
            deviations = getattr(target, "deviations", None)
            if deviations is None:
                return False
            if self._deviations is None:
                self._deviations = deviations.describe()
            for each in names:
                deviations.set(each, True)
            return True

        if not (yield turn_on):
            raise NotApplicable("the master in use cannot break a protocol rule")

    def conform(self) -> Plan[None]:
        """Turn off every deviation the check turned on."""
        yield self._conform

    def _conform(self, target: Any) -> None:
        deviations = getattr(target, "deviations", None)
        if deviations is None or self._deviations is None:
            return
        deviations.clear()
        if not self._isolated:
            for each, state in self._deviations.items():
                deviations.set(each, state)

    def isolate(self, target: Any) -> Any:
        """Begin a check that deviates: hold the master, with every deviation off.

        A deviation someone left on would damage the well-formed requests the
        check compares against, so each is turned off until the check ends.
        A master that sends on its own is held as well, so that no automatic
        task, repeated scan or other caller sends a request while one of the
        check's deviations is on, or between two of its steps. Returns what
        the master's own hold returns, to be awaited where it is awaitable.
        """

        def conform() -> None:
            deviations = getattr(target, "deviations", None)
            if deviations is not None and self._deviations is None:
                self._deviations = deviations.describe()
                self._isolated = True
                deviations.clear()

        reserve = getattr(target, "_reserve", None)
        if reserve is None:
            conform()
            return None

        async def held() -> None:
            # The deviations are turned off once the master is held, so that a
            # request another task has under way is finished as it was begun.
            await reserve()
            conform()

        return held()

    def restore(self, target: Any) -> None:
        """Put a master back as the check found it. Sends nothing.

        The deviations are set as they were before the check, and a master
        the check held is let go.
        """
        release = getattr(target, "_release", None)
        if release is not None:
            release()
        if self._deviations is None:
            return
        before, self._deviations, self._isolated = self._deviations, None, False
        target.deviations.clear()
        for each, state in before.items():
            target.deviations.set(each, state)

    def probe(self) -> Plan[bool]:
        """Send one small read, and return whether the outstation answered it."""
        if not self.served:
            raise NotApplicable("the outstation serves no point to read")
        kind, index = min(self.served, key=lambda each: (_PROBE_ORDER.index(each[0]), each[1]))
        readings: Readings = yield from read_plan([self.map.point(kind, index)])
        self.requests += len(readings.exchanges)
        return any(exchange.fragments for exchange in readings.exchanges)

    def unanswered(self, what: str) -> Plan[None]:
        """Send one small read, and fail the check if the outstation answers it."""
        answered = yield from self.probe()
        self.require(not answered, f"the outstation answered {what}")

    def answers(self, after: str, *, reads: int = RECOVERY_READS) -> Plan[None]:
        """Send small reads until one is answered, and fail the check if none of ``reads`` is."""
        for sent in range(1, reads + 1):
            if (yield from self.probe()):
                if sent > 1:
                    self.note(f"read {sent} after {after} was the first to be answered")
                return
        said = "a well-formed read" if reads == 1 else f"any of {reads} well-formed reads"
        raise Failed(f"the outstation did not answer {said} sent after {after}")

    def events(self) -> Plan[Exchange]:
        """Read the events of every class, and return the exchange."""
        exchange: Exchange = yield lambda target: target.scan("events")
        self._made([exchange], "a read of its events")
        return exchange

    def confirming(self) -> Plan[bool]:
        """Return whether the master sends a confirmation when a response asks for one."""
        confirming: bool = yield lambda target: bool(target.association.confirming)
        return confirming

    # ---------------------------------------------------------------- waiting

    def pause(self, seconds: float) -> Plan[None]:
        """Wait, sending nothing."""
        yield Pause(seconds)

    def eventually(self, attempt: Callable[[], Plan[T]]) -> Plan[T]:
        """Carry out ``attempt`` until it does not fail, for up to the settling time.

        For what an outstation takes time to do: a read that finds it not yet
        done is made again every half second. The last failure stands. A
        request that is not answered is not made again.
        """
        waited = 0.0
        while True:
            try:
                return (yield from attempt())
            except Unanswered:
                raise
            except Failed:
                if waited >= self.settle:
                    raise
            step = min(POLL_SECONDS, self.settle - waited)
            yield from self.pause(step)
            waited += step

    def becomes(self, point: Point, state: bool, said: str) -> Plan[None]:
        """Wait until a binary point reports ``state`` ONLINE, or fail with ``said``."""

        def attempt() -> Plan[None]:
            now = yield from self.state(point)
            self.require(now is state, f"{said}: {name(point)} did not become {state}")

        yield from self.eventually(attempt)

begin ¶

begin() -> None

Forget what the last check noted, counted and left to be put back.

Source code in src/py1815/master/bench.py

def begin(self) -> None:
    """Forget what the last check noted, counted and left to be put back."""
    self.title = None
    self.requests = 0
    self._notes = []
    self._afterwards = []
    self._deviations = None
    self._isolated = False

note ¶

note(text: str) -> None

Record something the report should say about this check.

Source code in src/py1815/master/bench.py

def note(self, text: str) -> None:
    """Record something the report should say about this check."""
    if text not in self._notes:
        self._notes.append(text)

require `staticmethod` ¶

require(holds: object, otherwise: str) -> None

Fail the check with otherwise unless holds.

Source code in src/py1815/master/bench.py

@staticmethod
def require(holds: object, otherwise: str) -> None:
    """Fail the check with ``otherwise`` unless ``holds``."""
    if not holds:
        raise Failed(otherwise)

afterwards ¶

afterwards(
    what: str, plan: Callable[[], Plan[Any]]
) -> None

Have plan carried out when the check ends, however it ends.

A check uses this to put back what it changed. The plans are carried out last one first.

Source code in src/py1815/master/bench.py

def afterwards(self, what: str, plan: Callable[[], Plan[Any]]) -> None:
    """Have ``plan`` carried out when the check ends, however it ends.

    A check uses this to put back what it changed. The plans are carried
    out last one first.
    """
    self._afterwards.append((what, plan))

put_back ¶

put_back() -> Plan[None]

Carry out what the check left to be done afterwards.

Source code in src/py1815/master/bench.py

def put_back(self) -> Plan[None]:
    """Carry out what the check left to be done afterwards."""
    while self._afterwards:
        what, plan = self._afterwards.pop()
        try:
            yield from plan()
        except Failed as failure:
            self.note(f"{what} could not be put back: {failure}")

supported ¶

supported(kind: Kind, index: int) -> bool

Return whether the outstation serves a point.

Source code in src/py1815/master/bench.py

def supported(self, kind: Kind, index: int) -> bool:
    """Return whether the outstation serves a point."""
    return (kind, index) in self.served

supported_in ¶

supported_in(
    kind: Kind, indices: Sequence[int]
) -> list[Point]

Return the points of one kind the outstation serves among indices.

Source code in src/py1815/master/bench.py

def supported_in(self, kind: Kind, indices: Sequence[int]) -> list[Point]:
    """Return the points of one kind the outstation serves among ``indices``."""
    return [self.map.point(kind, i) for i in indices if self.supported(kind, i)]

readback ¶

readback(output: Point) -> Point | None

Return the input that reads an output back, or None when none is served.

Source code in src/py1815/master/bench.py

def readback(self, output: Point) -> Point | None:
    """Return the input that reads an output back, or None when none is served."""
    mirror = self.profile.mirror(output)
    if mirror is None or mirror.address not in self.served:
        return None
    return mirror

read ¶

read(points: Sequence[Point]) -> Plan[Readings]

Read points in one request.

Source code in src/py1815/master/bench.py

def read(self, points: Sequence[Point]) -> Plan[Readings]:
    """Read points in one request."""
    readings: Readings = yield from read_plan(points)
    listed = ", ".join(address(point) for point in points[:4])
    more = f" and {len(points) - 4} more" if len(points) > 4 else ""
    self._made(readings.exchanges, f"a read of {listed}{more}")
    return readings

obtain ¶

obtain(point: Point) -> Plan[Reading]

Read one point and require the outstation to report it.

Source code in src/py1815/master/bench.py

def obtain(self, point: Point) -> Plan[Reading]:
    """Read one point and require the outstation to report it."""
    readings = yield from self.read([point])
    reading = readings.readings[0]
    self.require(reading.reported, f"{name(point)} could not be read")
    return reading

online ¶

online(point: Point) -> Plan[Reading]

Read a point and require it to be ONLINE with no other quality flag.

A variation that carries no flags says the same thing.

Source code in src/py1815/master/bench.py

def online(self, point: Point) -> Plan[Reading]:
    """Read a point and require it to be ONLINE with no other quality flag.

    A variation that carries no flags says the same thing.
    """
    reading = yield from self.obtain(point)
    flags = _flags(reading)
    self.require(
        flags is None or flags == AnalogQuality.ONLINE,
        f"{name(point)} is reported with flags 0x{flags or 0:02X}, and not as ONLINE alone",
    )
    return reading

offline ¶

offline(point: Point) -> Plan[Reading]

Read a point and require it to carry a value and no quality flag at all.

Source code in src/py1815/master/bench.py

def offline(self, point: Point) -> Plan[Reading]:
    """Read a point and require it to carry a value and no quality flag at all."""
    reading = yield from self.obtain(point)
    flags = _flags(reading)
    said = "no flags, which says ONLINE" if flags is None else f"flags 0x{flags:02X}"
    self.require(flags == 0, f"{name(point)} is reported with {said}, and not as offline")
    return reading

state ¶

state(point: Point) -> Plan[bool]

Return a binary point's state, which is required to be ONLINE.

Source code in src/py1815/master/bench.py

def state(self, point: Point) -> Plan[bool]:
    """Return a binary point's state, which is required to be ONLINE."""
    reading = yield from self.online(point)
    return bool(reading.raw)

value ¶

value(point: Point) -> Plan[float]

Return an analog point's value in engineering units, required to be ONLINE.

Source code in src/py1815/master/bench.py

def value(self, point: Point) -> Plan[float]:
    """Return an analog point's value in engineering units, required to be ONLINE."""
    reading = yield from self.online(point)
    return self._engineering(reading)

raw ¶

raw(point: Point) -> Plan[float]

Return an analog point's value in engineering units, whatever its flags say.

Source code in src/py1815/master/bench.py

def raw(self, point: Point) -> Plan[float]:
    """Return an analog point's value in engineering units, whatever its flags say."""
    reading = yield from self.obtain(point)
    return self._engineering(reading)

set ¶

set(output: Point | int, raw: int) -> Plan[CommandStatus]

Direct operate one analog output with a transmitted value. Return the status.

Source code in src/py1815/master/bench.py

def set(self, output: Point | int, raw: int) -> Plan[CommandStatus]:
    """Direct operate one analog output with a transmitted value. Return the status."""
    index = output if isinstance(output, int) else output.index
    status = yield from self._operate(
        [controls.analog(index, raw)], Mode.DIRECT, f"a write of {raw} to AO{index}"
    )
    return status

latch ¶

latch(output: Point | int, on: bool) -> Plan[CommandStatus]

Select and then operate a latch of one binary output. Return the status.

The status is the select's when the outstation refused the select.

Source code in src/py1815/master/bench.py

def latch(self, output: Point | int, on: bool) -> Plan[CommandStatus]:
    """Select and then operate a latch of one binary output. Return the status.

    The status is the select's when the outstation refused the select.
    """
    index = output if isinstance(output, int) else output.index
    status = yield from self._operate(
        [controls.binary(index, on)],
        Mode.SELECT,
        f"a latch {'on' if on else 'off'} of BO{index}",
    )
    return status

accepted ¶

accepted(status: CommandStatus, what: str) -> None

Fail the check unless the outstation accepted a control.

Source code in src/py1815/master/bench.py

def accepted(self, status: CommandStatus, what: str) -> None:
    """Fail the check unless the outstation accepted a control."""
    self.require(
        status is CommandStatus.SUCCESS, f"{what} was refused with status {status.name}"
    )

refused ¶

refused(
    status: CommandStatus, what: str, usual: CommandStatus
) -> None

Fail the check if the outstation accepted a control it is to refuse.

A refusal with a status other than usual is noted and not failed: which status refuses a control is the outstation's to choose.

Source code in src/py1815/master/bench.py

def refused(self, status: CommandStatus, what: str, usual: CommandStatus) -> None:
    """Fail the check if the outstation accepted a control it is to refuse.

    A refusal with a status other than ``usual`` is noted and not failed:
    which status refuses a control is the outstation's to choose.
    """
    self.require(status is not CommandStatus.SUCCESS, f"{what} was accepted")
    if status is not usual:
        self.note(f"{what} was refused with {status.name}, where {usual.name} is usual")

write_curve ¶

write_curve(
    number: int,
    *,
    type: int,
    x_units: int,
    y_units: int,
    points: Sequence[Sequence[int]],
) -> Plan[CurveWritten]

Write a curve in the profile's three steps and read it back.

Source code in src/py1815/master/bench.py

def write_curve(
    self,
    number: int,
    *,
    type: int,  # pylint: disable=redefined-builtin
    x_units: int,
    y_units: int,
    points: Sequence[Sequence[int]],
) -> Plan[CurveWritten]:
    """Write a curve in the profile's three steps and read it back."""
    written: CurveWritten = yield from write_curve_plan(
        self.profile, number, type=type, x_units=x_units, y_units=y_units, points=points
    )
    what = f"the write of curve {number}"
    for step in written.steps:
        self._made(step.exchanges, what)
    for read in written.readback.reads:
        self._made(read.exchanges, f"the read of curve {number}")
    return written

compare ¶

compare(document: str) -> Plan[Comparison]

Compare what the outstation serves with a Device Profile document.

Source code in src/py1815/master/bench.py

def compare(self, document: str) -> Plan[Comparison]:
    """Compare what the outstation serves with a Device Profile document."""
    comparison: Comparison = yield from compare_plan(document)
    self._made(comparison.exchanges, "the reads of the points its Device Profile declares")
    return comparison

deviate ¶

deviate(*names: str) -> Plan[None]

Have the master break the named protocol rules until the check ends.

The master's deviations are put back as the check found them when the check ends, or sooner by :meth:conform.

Source code in src/py1815/master/bench.py

def deviate(self, *names: str) -> Plan[None]:
    """Have the master break the named protocol rules until the check ends.

    The master's deviations are put back as the check found them when the
    check ends, or sooner by :meth:`conform`.
    """

    def turn_on(target: Any) -> bool:
        deviations = getattr(target, "deviations", None)
        if deviations is None:
            return False
        if self._deviations is None:
            self._deviations = deviations.describe()
        for each in names:
            deviations.set(each, True)
        return True

    if not (yield turn_on):
        raise NotApplicable("the master in use cannot break a protocol rule")

conform ¶

conform() -> Plan[None]

Turn off every deviation the check turned on.

Source code in src/py1815/master/bench.py

def conform(self) -> Plan[None]:
    """Turn off every deviation the check turned on."""
    yield self._conform

isolate ¶

isolate(target: Any) -> Any

Begin a check that deviates: hold the master, with every deviation off.

A deviation someone left on would damage the well-formed requests the check compares against, so each is turned off until the check ends. A master that sends on its own is held as well, so that no automatic task, repeated scan or other caller sends a request while one of the check's deviations is on, or between two of its steps. Returns what the master's own hold returns, to be awaited where it is awaitable.

Source code in src/py1815/master/bench.py

def isolate(self, target: Any) -> Any:
    """Begin a check that deviates: hold the master, with every deviation off.

    A deviation someone left on would damage the well-formed requests the
    check compares against, so each is turned off until the check ends.
    A master that sends on its own is held as well, so that no automatic
    task, repeated scan or other caller sends a request while one of the
    check's deviations is on, or between two of its steps. Returns what
    the master's own hold returns, to be awaited where it is awaitable.
    """

    def conform() -> None:
        deviations = getattr(target, "deviations", None)
        if deviations is not None and self._deviations is None:
            self._deviations = deviations.describe()
            self._isolated = True
            deviations.clear()

    reserve = getattr(target, "_reserve", None)
    if reserve is None:
        conform()
        return None

    async def held() -> None:
        # The deviations are turned off once the master is held, so that a
        # request another task has under way is finished as it was begun.
        await reserve()
        conform()

    return held()

restore ¶

restore(target: Any) -> None

Put a master back as the check found it. Sends nothing.

The deviations are set as they were before the check, and a master the check held is let go.

Source code in src/py1815/master/bench.py

def restore(self, target: Any) -> None:
    """Put a master back as the check found it. Sends nothing.

    The deviations are set as they were before the check, and a master
    the check held is let go.
    """
    release = getattr(target, "_release", None)
    if release is not None:
        release()
    if self._deviations is None:
        return
    before, self._deviations, self._isolated = self._deviations, None, False
    target.deviations.clear()
    for each, state in before.items():
        target.deviations.set(each, state)

probe ¶

probe() -> Plan[bool]

Send one small read, and return whether the outstation answered it.

Source code in src/py1815/master/bench.py

def probe(self) -> Plan[bool]:
    """Send one small read, and return whether the outstation answered it."""
    if not self.served:
        raise NotApplicable("the outstation serves no point to read")
    kind, index = min(self.served, key=lambda each: (_PROBE_ORDER.index(each[0]), each[1]))
    readings: Readings = yield from read_plan([self.map.point(kind, index)])
    self.requests += len(readings.exchanges)
    return any(exchange.fragments for exchange in readings.exchanges)

unanswered ¶

unanswered(what: str) -> Plan[None]

Send one small read, and fail the check if the outstation answers it.

Source code in src/py1815/master/bench.py

def unanswered(self, what: str) -> Plan[None]:
    """Send one small read, and fail the check if the outstation answers it."""
    answered = yield from self.probe()
    self.require(not answered, f"the outstation answered {what}")

answers ¶

answers(
    after: str, *, reads: int = RECOVERY_READS
) -> Plan[None]

Send small reads until one is answered, and fail the check if none of reads is.

Source code in src/py1815/master/bench.py

def answers(self, after: str, *, reads: int = RECOVERY_READS) -> Plan[None]:
    """Send small reads until one is answered, and fail the check if none of ``reads`` is."""
    for sent in range(1, reads + 1):
        if (yield from self.probe()):
            if sent > 1:
                self.note(f"read {sent} after {after} was the first to be answered")
            return
    said = "a well-formed read" if reads == 1 else f"any of {reads} well-formed reads"
    raise Failed(f"the outstation did not answer {said} sent after {after}")

events ¶

events() -> Plan[Exchange]

Read the events of every class, and return the exchange.

Source code in src/py1815/master/bench.py

def events(self) -> Plan[Exchange]:
    """Read the events of every class, and return the exchange."""
    exchange: Exchange = yield lambda target: target.scan("events")
    self._made([exchange], "a read of its events")
    return exchange

confirming ¶

confirming() -> Plan[bool]

Return whether the master sends a confirmation when a response asks for one.

Source code in src/py1815/master/bench.py

def confirming(self) -> Plan[bool]:
    """Return whether the master sends a confirmation when a response asks for one."""
    confirming: bool = yield lambda target: bool(target.association.confirming)
    return confirming

pause ¶

pause(seconds: float) -> Plan[None]

Wait, sending nothing.

Source code in src/py1815/master/bench.py

def pause(self, seconds: float) -> Plan[None]:
    """Wait, sending nothing."""
    yield Pause(seconds)

eventually ¶

eventually(attempt: Callable[[], Plan[T]]) -> Plan[T]

Carry out attempt until it does not fail, for up to the settling time.

For what an outstation takes time to do: a read that finds it not yet done is made again every half second. The last failure stands. A request that is not answered is not made again.

Source code in src/py1815/master/bench.py

def eventually(self, attempt: Callable[[], Plan[T]]) -> Plan[T]:
    """Carry out ``attempt`` until it does not fail, for up to the settling time.

    For what an outstation takes time to do: a read that finds it not yet
    done is made again every half second. The last failure stands. A
    request that is not answered is not made again.
    """
    waited = 0.0
    while True:
        try:
            return (yield from attempt())
        except Unanswered:
            raise
        except Failed:
            if waited >= self.settle:
                raise
        step = min(POLL_SECONDS, self.settle - waited)
        yield from self.pause(step)
        waited += step

becomes ¶

becomes(point: Point, state: bool, said: str) -> Plan[None]

Wait until a binary point reports state ONLINE, or fail with said.

Source code in src/py1815/master/bench.py

def becomes(self, point: Point, state: bool, said: str) -> Plan[None]:
    """Wait until a binary point reports ``state`` ONLINE, or fail with ``said``."""

    def attempt() -> Plan[None]:
        now = yield from self.state(point)
        self.require(now is state, f"{said}: {name(point)} did not become {state}")

    yield from self.eventually(attempt)

name ¶

name(point: Point) -> str

Return a point's address and name, as in AO87 (Charging limit).

Source code in src/py1815/master/bench.py

def name(point: Point) -> str:
    """Return a point's address and name, as in ``AO87 (Charging limit)``."""
    return f"{address(point)} ({label(point)})"

run_plan ¶

run_plan(
    bench: Bench,
    checks: Sequence[Check],
    *,
    commanding: bool,
    deviating: bool = False,
    on_result: Callable[[Result], None] | None = None,
) -> Plan[tuple[Result, ...]]

Carry out checks in order, and return how each ended.

The run begins by reading which points the outstation serves, with every deviation of the master off. A check that writes is not run unless commanding is set, and a check that has the master break a protocol rule is not run unless deviating is set. When the connection is lost the check under way fails and the rest are not run. on_result is called with each result as its check ends.

Raises :class:NoAnswer when the outstation does not answer the first read.

Source code in src/py1815/master/bench.py

def run_plan(
    bench: Bench,
    checks: Sequence[Check],
    *,
    commanding: bool,
    deviating: bool = False,
    on_result: Callable[[Result], None] | None = None,
) -> Plan[tuple[Result, ...]]:
    """Carry out checks in order, and return how each ended.

    The run begins by reading which points the outstation serves, with every
    deviation of the master off. A check
    that writes is not run unless ``commanding`` is set, and a check that has
    the master break a protocol rule is not run unless ``deviating`` is set.
    When the connection is lost the check under way fails and the rest are
    not run. ``on_result`` is called with each result as its check ends.

    Raises :class:`NoAnswer` when the outstation does not answer the first read.
    """
    wanted = [(POINT_TYPES[point.kind], point.index) for point in bench.map.points.values()]
    # What the outstation serves is read with every deviation off: a
    # deviation someone left on would make a served point look absent.
    yield bench.isolate
    lost: OSError | None = None
    survey = None
    try:
        survey = yield from survey_plan(wanted)
    except _LOST as error:
        lost = error
    yield bench.restore
    if survey is None:
        raise NoAnswer(f"the connection was lost: {lost}") from lost
    if not survey.exchanges[0].fragments:
        raise NoAnswer("the outstation did not answer a class 0 read")
    if any(not exchange.fragments for exchange in survey.exchanges):
        # A point missing because a read went unanswered would be taken as
        # a point the outstation does not serve.
        raise NoAnswer("the outstation did not answer every read of the points it serves")
    kinds = {point_type: kind for kind, point_type in POINT_TYPES.items()}
    bench.served = frozenset(
        (kinds[point_type], index) for point_type, index in survey.served if point_type in kinds
    )

    run = _Run()
    for check in checks:
        if run.lost is not None:
            result = Result(check.id, check.title, Verdict.NOT_RUN, run.lost)
        elif check.commands and not commanding:
            result = Result(
                check.id,
                check.title,
                Verdict.NOT_RUN,
                "it writes to the outstation, and the run was not allowed to",
            )
        elif check.deviates and not deviating:
            result = Result(
                check.id,
                check.title,
                Verdict.NOT_RUN,
                "it breaks a protocol rule on purpose, and the run was not allowed to",
            )
        else:
            result = yield from _one(bench, check, run)
        run.results.append(result)
        if on_result is not None:
            on_result(result)
    return tuple(run.results)

drive ¶

drive(
    target: Any,
    plan: Plan[T],
    pause: Callable[[float], None],
) -> T

Carry out a plan on a master whose requests return at once, as a Loopback's do.

pause is called with the seconds of each wait in the plan.

Source code in src/py1815/master/bench.py

def drive(target: Any, plan: Plan[T], pause: Callable[[float], None]) -> T:
    """Carry out a plan on a master whose requests return at once, as a Loopback's do.

    ``pause`` is called with the seconds of each wait in the plan.
    """
    try:
        step = next(plan)
        while True:
            try:
                answer: Any = None
                if isinstance(step, Pause):
                    pause(step.seconds)
                else:
                    answer = step(target)
            except _LOST as error:
                step = plan.throw(error)
            else:
                step = plan.send(answer)
    except StopIteration as done:
        return done.value  # type: ignore[no-any-return]

drive_async `async` ¶

drive_async(
    target: Any,
    plan: Plan[T],
    pause: Callable[[float], Awaitable[None]],
) -> T

Carry out a plan on a master whose requests are awaited, as an Outstation's are.

pause is awaited with the seconds of each wait in the plan.

Source code in src/py1815/master/bench.py

async def drive_async(target: Any, plan: Plan[T], pause: Callable[[float], Awaitable[None]]) -> T:
    """Carry out a plan on a master whose requests are awaited, as an Outstation's are.

    ``pause`` is awaited with the seconds of each wait in the plan.
    """
    try:
        step = next(plan)
        while True:
            try:
                answer: Any = None
                if isinstance(step, Pause):
                    await pause(step.seconds)
                else:
                    answer = step(target)
                    if inspect.isawaitable(answer):
                        answer = await answer
            except _LOST as error:
                step = plan.throw(error)
            else:
                step = plan.send(answer)
    except StopIteration as done:
        return done.value  # type: ignore[no-any-return]

py1815.master.checks ¶

The checks py1815-master evaluate runs against an outstation.

The DER profile set carries out EPRI's Test Procedure for Validating DNP Application Note AN2018-001 in Distributed Energy Resources (3002016144) the way a controlling station would: read a function's points, write its settings and read them back, enable and disable it, and edit curves through the block all curves share. Each check is named for the identifier the report gives its procedure: MON-001 is the monitoring procedure. The report is EPRI's and is not reproduced; the steps are described here in our own words. Where the report and IEEE 1815.2 differ, these follow the standard.

The procedures carry no point list of their own. Which input reads an output back, which points make up a function and which input says it is supported come from the profile tables, and a point counts as supported when the outstation reports it. A function the outstation says it does not support is checked for that: it cannot be enabled and none of its points are served.

A check leaves the outstation as it found it where that is one command: a switch, a permission, started or stopped. Settings, curves and which functions are enabled are left as the check last wrote them, with every function it enabled disabled again.

The deviation set has the master break a protocol rule on purpose (:mod:py1815.master.deviations) and requires what the standard requires of the outstation: silence for a frame or a segment it has to discard, and events kept until the right confirmation arrives. These checks write nothing to the outstation. They are run only when the run is allowed to deviate.

Check `dataclass` ¶

One named procedure.

Source code in src/py1815/master/bench.py

@dataclass(frozen=True)
class Check:
    """One named procedure."""

    #: A short identifier, as in ``MON-001``.
    id: str
    title: str
    #: Whether the check writes to the outstation.
    commands: bool
    procedure: Callable[[Bench], Plan[None]]
    #: The set the check belongs to.
    group: str = "der"
    #: Whether the check has the master break a protocol rule on purpose.
    deviates: bool = False

SampleCurve `dataclass` ¶

A valid curve of one type, to write to an outstation.

Source code in src/py1815/master/checks.py

@dataclass(frozen=True)
class SampleCurve:
    """A valid curve of one type, to write to an outstation."""

    type: int
    x_units: int
    y_units: int
    points: tuple[tuple[int, int], ...]

valid_values ¶

valid_values(point: Point) -> tuple[int, int]

Return two transmitted values inside the range the tables give an output.

They differ unless the range holds one value.

Source code in src/py1815/master/checks.py

def valid_values(point: Point) -> tuple[int, int]:
    """Return two transmitted values inside the range the tables give an output.

    They differ unless the range holds one value.
    """
    low = int(point.minimum) if point.minimum is not None else 0
    high = int(point.maximum) if point.maximum is not None else low + 100
    if high < low:
        low = high - 100
    span = min(high - low, 400)
    if span < 1:
        return low, low
    if span == 1:
        return low, high
    return low + max(1, span // 4), low + max(2, span // 2)

check_settings ¶

check_settings(
    bench: Bench,
    outputs: Sequence[Point],
    which: int,
    *,
    enabled: bool = True,
) -> Plan[None]

Write one set of valid values to analog outputs, and read each back.

The value read back is compared in engineering units, since the tables scale an output and the input that reads it back independently. With enabled false the readbacks belong to a function that is disabled: they are required to carry the value and no ONLINE flag.

Source code in src/py1815/master/checks.py

def check_settings(
    bench: Bench, outputs: Sequence[Point], which: int, *, enabled: bool = True
) -> Plan[None]:
    """Write one set of valid values to analog outputs, and read each back.

    The value read back is compared in engineering units, since the tables
    scale an output and the input that reads it back independently. With
    ``enabled`` false the readbacks belong to a function that is disabled:
    they are required to carry the value and no ONLINE flag.
    """
    for output in outputs:
        raw = valid_values(output)[which]
        status = yield from bench.set(output, raw)
        bench.accepted(status, f"a write of {raw} to {name(output)}")
    for output in outputs:
        back = bench.readback(output)
        if back is None:
            bench.note(f"no input reads {address(output)} back, so its value was not verified")
            continue
        expected = output.from_wire(valid_values(output)[which])

        def attempt(
            output: Point = output, back: Point = back, expected: float = expected
        ) -> Plan[None]:
            reading = yield from (bench.online(back) if enabled else bench.offline(back))
            raw = reading.raw
            if raw is None or isinstance(raw, bool):
                raise Failed(f"{name(back)} did not report a number")
            read = back.from_wire(raw)
            bench.require(
                abs(read - expected) <= _tolerance(output, back),
                f"{name(back)} reads {read:g}, and {address(output)} was set to {expected:g}",
            )

        yield from bench.eventually(attempt)

monitoring ¶

monitoring(bench: Bench) -> Plan[None]

Check that every served point of the system meter is ONLINE and in range.

Source code in src/py1815/master/checks.py

def monitoring(bench: Bench) -> Plan[None]:
    """Check that every served point of the system meter is ONLINE and in range."""
    supported = bench.supported_in(Kind.AI, METER)
    if not supported:
        raise NotApplicable("the outstation serves no point of the system meter")
    bench.note(f"{len(supported)} of the meter's {len(METER)} points are served")
    for point in supported:
        reading = yield from bench.online(point)
        raw = reading.raw
        bench.require(
            isinstance(raw, (int, float)) and not isinstance(raw, bool) and point.in_range(raw),
            f"{name(point)} reads {raw}, outside the range the tables give",
        )

alarms ¶

alarms(bench: Bench) -> Plan[None]

Check that every served alarm is ONLINE, and note the ones that are raised.

Source code in src/py1815/master/checks.py

def alarms(bench: Bench) -> Plan[None]:
    """Check that every served alarm is ONLINE, and note the ones that are raised."""
    supported = [bench.map.point(Kind.BI, i) for i in ALARMS if bench.supported(Kind.BI, i)]
    if not supported:
        raise NotApplicable("the outstation serves none of the alarm points")
    bench.note(f"{len(supported)} of the {len(ALARMS)} alarm points are served")
    raised = []
    for point in supported:
        if (yield from bench.state(point)):
            raised.append(address(point))
    if raised:
        bench.note(f"raised when read: {', '.join(raised)}")

operating_states ¶

operating_states(bench: Bench) -> Plan[None]

Check that every served operating state is ONLINE and that the states agree.

Source code in src/py1815/master/checks.py

def operating_states(bench: Bench) -> Plan[None]:
    """Check that every served operating state is ONLINE and that the states agree."""
    supported = bench.supported_in(Kind.BI, STATES)
    if not supported:
        raise NotApplicable("the outstation serves none of the operating state points")
    states: dict[int, bool] = {}
    for point in supported:
        states[point.index] = yield from bench.state(point)
    if BI_STARTED in states and BI_STOPPED in states:
        bench.require(
            not (states[BI_STARTED] and states[BI_STOPPED]),
            f"BI{BI_STARTED} and BI{BI_STOPPED} report started and stopped at once",
        )
    held = [f"BI{index}" for index in EXCLUSIVE_STATES if states.get(index)]
    block = f"BI{EXCLUSIVE_STATES[0]} to BI{EXCLUSIVE_STATES[-1]}"
    bench.require(
        len(held) <= 1, f"no more than one of {block} may be set, and {', '.join(held)} are"
    )
    if all(index in states for index in EXCLUSIVE_STATES):
        # With the whole block served, the DER is in one of the states.
        bench.require(held, f"one of {block} is to be set, and none is")

connect_and_disconnect ¶

connect_and_disconnect(bench: Bench) -> Plan[None]

Check that the connect settings read back and the switch follows its command.

Source code in src/py1815/master/checks.py

def connect_and_disconnect(bench: Bench) -> Plan[None]:
    """Check that the connect settings read back and the switch follows its command."""
    switch = bench.map.get(Kind.BO, der.BO_CONNECT)
    closed = None if switch is None else bench.readback(switch)
    if switch is None or closed is None:
        raise NotApplicable("the outstation serves no status of the connect switch")
    settings = bench.supported_in(Kind.AO, CONNECT_SETTINGS)
    for output in settings:
        back = bench.readback(output)
        if back is not None:
            yield from bench.online(back)
    if bench.supported(Kind.BI, BI_SWITCH_MOVING):
        yield from bench.online(bench.map.point(Kind.BI, BI_SWITCH_MOVING))

    yield from check_settings(bench, settings, 0)
    yield from check_settings(bench, settings, 1)
    yield from _toggle(bench, switch, closed, "the connect switch did not follow its command")

service ¶

service(bench: Bench) -> Plan[None]

Check that the service settings read back, then stop and start the DER.

Source code in src/py1815/master/checks.py

def service(bench: Bench) -> Plan[None]:
    """Check that the service settings read back, then stop and start the DER."""
    points = _service(bench)
    settings = bench.supported_in(Kind.AO, SERVICE_SETTINGS)
    for output in (*settings, points.start, points.stop, points.may_start, points.may_stop):
        back = bench.readback(output)
        if back is not None:
            yield from bench.online(back)

    yield from check_settings(bench, settings, 0)
    yield from check_settings(bench, settings, 1)

    for permission in (points.may_stop, points.may_start):
        back = bench.readback(permission)
        if back is None:
            bench.note(f"no input reads {address(permission)} back, so it was not exercised")
            continue
        yield from _toggle(bench, permission, back, "a permission did not follow its command")

    started = yield from bench.state(points.started)
    _restore_run(bench, points, started)
    for target in (not started, started):
        yield from _run_to(bench, points, target)
        if not target:
            # The input that reads the stop command back says the DER is stopped.
            back = bench.readback(points.stop)
            if back is not None:
                yield from bench.becomes(back, True, "the stop is not read back")

start_without_permission ¶

start_without_permission(bench: Bench) -> Plan[None]

Check that a start is refused without permission and the DER stays stopped.

Source code in src/py1815/master/checks.py

def start_without_permission(bench: Bench) -> Plan[None]:
    """Check that a start is refused without permission and the DER stays stopped."""
    points = _service(bench)
    started = yield from bench.state(points.started)
    _restore_run(bench, points, started)
    if started:
        yield from _run_to(bench, points, False)
    yield from _withdraw(bench, points, points.may_start, started)
    status = yield from bench.latch(points.start, True)
    bench.refused(status, "a start without permission", CommandStatus.BLOCKED)
    yield from bench.pause(min(bench.settle, 3.0))
    bench.require(
        (yield from bench.state(points.stopped)), "the DER did not stay stopped after the refusal"
    )

stop_without_permission ¶

stop_without_permission(bench: Bench) -> Plan[None]

Check that a stop is refused without permission and the DER stays started.

Source code in src/py1815/master/checks.py

def stop_without_permission(bench: Bench) -> Plan[None]:
    """Check that a stop is refused without permission and the DER stays started."""
    points = _service(bench)
    started = yield from bench.state(points.started)
    _restore_run(bench, points, started)
    if not started:
        yield from _run_to(bench, points, True)
    yield from _withdraw(bench, points, points.may_stop, started)
    status = yield from bench.latch(points.stop, True)
    bench.refused(status, "a stop without permission", CommandStatus.BLOCKED)
    bench.require(
        not (yield from bench.state(points.stopping)), "a stop is under way after the refusal"
    )
    yield from bench.pause(min(bench.settle, 3.0))
    bench.require(
        (yield from bench.state(points.started)), "the DER did not stay started after the refusal"
    )
    bench.require(not (yield from bench.state(points.stopped)), "the DER stopped after the refusal")

curve_reference ¶

curve_reference(bench: Bench) -> Plan[None]

Check that the referenced indicator is set only for a curve a function names.

Source code in src/py1815/master/checks.py

def curve_reference(bench: Bench) -> Plan[None]:
    """Check that the referenced indicator is set only for a curve a function names."""
    block = _block(bench)
    yield from block.release()
    bench.afterwards("which curves the functions name", block.release)
    for index in block.settings:
        back = bench.readback(bench.map.point(Kind.AO, index))
        if back is not None:
            named = yield from bench.raw(back)
            bench.require(named == 0, f"{name(back)} reads {named:g} after it was set to 0")

    total = yield from block.count()
    looked_at = 2 if total is None else min(total, 2)
    for number in range(1, looked_at + 1):
        yield from block.show(number)
        bench.require(
            not (yield from bench.state(block.referenced)),
            f"curve {number} is reported as referenced, and no function names it",
        )

    index, samples = next(iter(block.settings.items()))
    yield from block.define(1, samples[0])
    bench.require(
        not (yield from bench.state(block.referenced)),
        "curve 1 is reported as referenced once defined, and no function names it",
    )
    status = yield from block.point_to(index, 1)
    bench.accepted(status, f"a write of 1 to AO{index}")
    bench.require(
        (yield from bench.state(block.referenced)),
        f"curve 1 is not reported as referenced, and AO{index} names it",
    )

    # With another curve selected, the indicator is about that one.
    if looked_at >= 2:
        yield from block.show(2)
        bench.require(
            not (yield from bench.state(block.referenced)),
            "curve 2 is reported as referenced while only curve 1 is named",
        )
        yield from block.show(1)
    status = yield from block.point_to(index, 0)
    bench.accepted(status, f"a write of 0 to AO{index}")
    bench.require(
        not (yield from bench.state(block.referenced)),
        "curve 1 is still reported as referenced after the function let it go",
    )

curve_that_does_not_exist ¶

curve_that_does_not_exist(bench: Bench) -> Plan[None]

Check that a curve beyond the last one stored cannot be selected.

Source code in src/py1815/master/checks.py

def curve_that_does_not_exist(bench: Bench) -> Plan[None]:
    """Check that a curve beyond the last one stored cannot be selected."""
    block = _block(bench)
    total = yield from block.count()
    if total is None:
        raise NotApplicable(
            f"the outstation accepted every curve number up to {MAX_CURVES_PROBED}; "
            "say how many it stores"
        )
    if total < 1:
        raise NotApplicable("the outstation stores no curve")
    if bench.curves is None:
        bench.note(f"the outstation stores {total} curves, found by selecting each")
    yield from block.show(total)
    before = yield from bench.value(block.selected)
    status = yield from block.select(total + 1)
    bench.refused(status, f"the selection of curve {total + 1}", CommandStatus.OUT_OF_RANGE)
    after = yield from bench.value(block.selected)
    bench.require(after == before, f"{name(block.selected)} changed to {after:g} after a refusal")

curve_locking ¶

curve_locking(bench: Bench) -> Plan[None]

Check that a curve named by an enabled function cannot be edited.

Source code in src/py1815/master/checks.py

def curve_locking(bench: Bench) -> Plan[None]:
    """Check that a curve named by an enabled function cannot be edited."""
    block = _block(bench)
    index, sample, _other, function, enabled = _lock_target(bench, block)
    total = yield from block.count()
    if total is not None and total < 2:
        raise NotApplicable("the outstation stores fewer than two curves")
    locked, free = 1, 2

    yield from block.release()
    bench.afterwards("which curves the functions name", block.release)
    yield from block.define(locked, sample)
    status = yield from block.point_to(index, locked)
    bench.accepted(status, f"a write of {locked} to AO{index}")
    _disable_afterwards(bench, function)
    status = yield from bench.latch(function.enable, True)
    bench.accepted(status, f"the enabling of {function.name}")
    yield from bench.becomes(enabled, True, f"{function.name} was not enabled")

    yield from block.show(locked)
    bench.require(
        (yield from bench.state(block.referenced)),
        f"curve {locked} is not reported as referenced while {function.name} names it",
    )
    watched = min(_WATCHED_POINTS, len(sample.points))
    fields = yield from block.fields()
    before = yield from block.points(watched)

    first_x = block.outputs[curves.FIELDS]
    first_y = block.outputs[curves.FIELDS + 1]
    for output, value in ((first_x, before[0] + 10), (first_y, before[1] + 10)):
        status = yield from bench.set(output, int(value))
        bench.refused(
            status,
            f"a write to a point of the locked curve {locked}",
            CommandStatus.AUTOMATION_INHIBIT,
        )
    for output in block.outputs[: curves.FIELDS]:
        status = yield from bench.set(output, 1)
        bench.refused(
            status,
            f"a write to a field of the locked curve {locked}",
            CommandStatus.AUTOMATION_INHIBIT,
        )
    yield from block.unchanged(fields, before, "while it was locked")

    # Another curve is not locked by this one being in use.
    yield from block.show(free)
    status = yield from bench.set(first_x, 123)
    bench.accepted(status, f"a write to a point of curve {free}, which no function names")

    # Disabling the function releases the lock.
    status = yield from bench.latch(function.enable, False)
    bench.accepted(status, f"the disabling of {function.name}")
    yield from bench.becomes(enabled, False, f"{function.name} was not disabled")
    yield from block.show(locked)
    status = yield from bench.set(first_x, int(before[0] + 10))
    bench.accepted(
        status, f"a write to a point of curve {locked} once {function.name} was disabled"
    )
    now = yield from block.points(1)
    bench.require(now[0] == before[0] + 10, f"curve {locked} does not read back the write")

curve_moved_under_function ¶

curve_moved_under_function(bench: Bench) -> Plan[None]

Check that an enabled function can move to another curve, freeing the first.

Source code in src/py1815/master/checks.py

def curve_moved_under_function(bench: Bench) -> Plan[None]:
    """Check that an enabled function can move to another curve, freeing the first."""
    block = _block(bench)
    index, first, second, function, enabled = _lock_target(bench, block)
    total = yield from block.count()
    if total is not None and total < 2:
        raise NotApplicable("the outstation stores fewer than two curves")

    yield from block.release()
    bench.afterwards("which curves the functions name", block.release)
    yield from block.define(1, first)
    yield from block.define(2, second)
    status = yield from block.point_to(index, 1)
    bench.accepted(status, f"a write of 1 to AO{index}")
    _disable_afterwards(bench, function)
    status = yield from bench.latch(function.enable, True)
    bench.accepted(status, f"the enabling of {function.name}")
    yield from bench.becomes(enabled, True, f"{function.name} was not enabled")
    status = yield from block.point_to(index, 2)
    bench.accepted(status, f"a write of 2 to AO{index} while {function.name} is enabled")

    first_x = block.outputs[curves.FIELDS]
    yield from block.show(1)
    status = yield from bench.set(first_x, 111)
    bench.accepted(status, "a write to a point of curve 1, which the function left")
    yield from block.show(2)
    fields = yield from block.fields()
    before = yield from block.points(min(_WATCHED_POINTS, len(second.points)))
    status = yield from bench.set(first_x, 111)
    bench.refused(
        status,
        "a write to a point of curve 2, which the function now names",
        CommandStatus.AUTOMATION_INHIBIT,
    )
    yield from block.unchanged(fields, before, "after a refused write")

curve_type_mismatch ¶

curve_type_mismatch(bench: Bench) -> Plan[None]

Check that a function cannot name a curve of a type it does not follow.

Source code in src/py1815/master/checks.py

def curve_type_mismatch(bench: Bench) -> Plan[None]:
    """Check that a function cannot name a curve of a type it does not follow."""
    block = _block(bench)
    kinds = {index: samples[0] for index, samples in block.settings.items()}
    pairs = [
        (one, other) for one in kinds for other in kinds if kinds[one].type != kinds[other].type
    ]
    if not pairs:
        raise NotApplicable(
            "the outstation serves no two settings that name curves of different types"
        )
    first, second = pairs[0]
    back = bench.readback(bench.map.point(Kind.AO, second))

    yield from block.release()
    bench.afterwards("which curves the functions name", block.release)
    yield from block.define(1, kinds[first])

    status = yield from block.point_to(second, 1)
    bench.refused(
        status, f"pointing AO{second} at a curve of another type", CommandStatus.NOT_SUPPORTED
    )
    if back is not None:
        named = yield from bench.raw(back)
        bench.require(named == 0, f"{name(back)} reads {named:g} after the refusal")
    status = yield from block.point_to(first, 1)
    bench.accepted(status, f"pointing AO{first} at a curve of its own type")

    # Nor can a curve be turned into another type under a function that names it.
    yield from block.show(1)
    type_field = block.outputs[curves.TYPE]
    fields = yield from block.fields()
    status = yield from bench.set(type_field, kinds[second].type)
    bench.refused(
        status, "changing the type of a curve a function names", CommandStatus.NOT_SUPPORTED
    )
    yield from block.unchanged(fields, [], "after a refused change of its type")
    status = yield from block.point_to(first, 0)
    bench.accepted(status, f"a write of 0 to AO{first}")
    status = yield from bench.set(type_field, kinds[second].type)
    bench.accepted(status, "changing the type of a curve no function names")

curve_not_defined ¶

curve_not_defined(bench: Bench) -> Plan[None]

Check that a function cannot name a curve with no type, or one that does not exist.

Source code in src/py1815/master/checks.py

def curve_not_defined(bench: Bench) -> Plan[None]:
    """Check that a function cannot name a curve with no type, or one that does not exist."""
    block = _block(bench)
    index = next(iter(block.settings))
    total = yield from block.count()
    if total is None:
        raise NotApplicable(
            f"the outstation accepted every curve number up to {MAX_CURVES_PROBED}; "
            "say how many it stores"
        )
    yield from block.release()
    bench.afterwards("which curves the functions name", block.release)

    undefined = None
    for number in range(total, 0, -1):
        yield from block.show(number)
        kind = yield from bench.raw(block.inputs[curves.TYPE])
        if kind == 0:
            undefined = number
            break
    if undefined is None:
        bench.note("every curve the outstation stores has a type, so none was undefined")
    else:
        status = yield from block.point_to(index, undefined)
        bench.refused(
            status,
            f"pointing AO{index} at curve {undefined}, which has no type",
            CommandStatus.NOT_SUPPORTED,
        )
    status = yield from block.point_to(index, total + 1)
    bench.refused(
        status,
        f"pointing AO{index} at curve {total + 1}, which does not exist",
        CommandStatus.OUT_OF_RANGE,
    )

mode_check ¶

mode_check(enable: int) -> Callable[[Bench], Plan[None]]

Return the mode procedure for the function a binary output enables.

Source code in src/py1815/master/checks.py

def mode_check(enable: int) -> Callable[[Bench], Plan[None]]:
    """Return the mode procedure for the function a binary output enables."""

    def procedure(bench: Bench) -> Plan[None]:
        function = next(
            (each for each in bench.profile.functions if each.enable.index == enable), None
        )
        if function is None:
            raise NotApplicable(f"the tables have no function enabled by BO{enable}")
        if function.supports.address not in bench.served:
            raise NotApplicable(
                f"the outstation does not serve {name(function.supports)}, which says whether "
                f"it supports {function.name}"
            )
        supported = yield from bench.state(function.supports)
        bench.title = f"{function.name}: {'supported' if supported else 'not supported'}"
        described = _mode(bench, function)
        if supported:
            yield from _supported(bench, described)
        else:
            yield from _unsupported(bench, described)

    return procedure

device_profile ¶

device_profile(bench: Bench) -> Plan[None]

Check that the outstation serves the points its Device Profile declares, and no others.

Source code in src/py1815/master/checks.py

def device_profile(bench: Bench) -> Plan[None]:
    """Check that the outstation serves the points its Device Profile declares, and no others."""
    if bench.device_profile is None:
        raise NotApplicable("no Device Profile document was given")
    comparison = yield from bench.compare(bench.device_profile)
    bench.note(f"{len(comparison.served)} of {len(comparison.declared)} declared points are served")

    kinds = {point_type: kind.value for kind, point_type in POINT_TYPES.items()}

    def where(point_type: PointType, index: int) -> str:
        return f"{kinds.get(point_type, point_type.value + ' ')}{index}"

    def some(items: Sequence[str]) -> str:
        return ", ".join(items[:8]) + (f" and {len(items) - 8} more" if len(items) > 8 else "")

    problems = []
    if comparison.absent:
        absent = [where(each.type, each.index) for each in comparison.absent]
        problems.append(f"declared and not served: {some(absent)}")
    if comparison.undeclared:
        undeclared = [where(kind, index) for kind, index in comparison.undeclared]
        problems.append(f"served and not declared: {some(undeclared)}")
    if comparison.class_0:
        misplaced = [
            f"{where(each.type, each.index)} ({'in' if carried else 'not in'} class 0)"
            for each, carried in comparison.class_0
        ]
        problems.append(f"class 0 is not as declared: {some(misplaced)}")
    bench.require(not problems, "; ".join(problems))

not_answered ¶

not_answered(
    deviation: str,
    what: str,
    *,
    reads: int = RECOVERY_READS,
) -> Callable[[Bench], Plan[None]]

Return the check that a request sent with one deviation on draws no answer.

A well-formed read is answered first, so that the silence that follows is about the deviation. With the deviation on, a read has to go unanswered. With it off again, one of the next reads reads has to be answered, so the silence was about that one request and not a receiver that stopped.

What makes silence the required answer depends on the deviation:

A frame that fails a checksum, is cut short, or claims a length it does not carry is discarded by the data link layer and nothing is sent for it (IEEE 1815-2012 9.2.4.3). The invalid CRC procedure of section 6.6.2 of the IED Certification Procedure tests the two checksum cases.
A segment that is not a first segment and follows no series in progress is discarded, and a new first segment abandons a series that did not end (IEEE 1815-2012 8.2.4, Table 8-1, as TB2013-003 restates it).

Source code in src/py1815/master/checks.py

def not_answered(
    deviation: str, what: str, *, reads: int = RECOVERY_READS
) -> Callable[[Bench], Plan[None]]:
    """Return the check that a request sent with one deviation on draws no answer.

    A well-formed read is answered first, so that the silence that follows is
    about the deviation. With the deviation on, a read has to go unanswered.
    With it off again, one of the next ``reads`` reads has to be answered, so
    the silence was about that one request and not a receiver that stopped.

    What makes silence the required answer depends on the deviation:

    - A frame that fails a checksum, is cut short, or claims a length it does
      not carry is discarded by the data link layer and nothing is sent for it
      (IEEE 1815-2012 9.2.4.3). The invalid CRC procedure of section 6.6.2 of
      the IED Certification Procedure tests the two checksum cases.
    - A segment that is not a first segment and follows no series in progress
      is discarded, and a new first segment abandons a series that did not
      end (IEEE 1815-2012 8.2.4, Table 8-1, as TB2013-003 restates it).
    """

    def procedure(bench: Bench) -> Plan[None]:
        yield from bench.answers("the check began", reads=1)
        yield from bench.deviate(deviation)
        yield from bench.unanswered(what)
        yield from bench.conform()
        yield from bench.answers(what, reads=reads)

    return procedure

events_kept ¶

events_kept(
    deviation: str, what: str
) -> Callable[[Bench], Plan[None]]

Return the check that events are kept until the right confirmation arrives.

The events are read with one confirmation deviation on, so the outstation does not receive the confirmation it asked for. Read again, it has to report every one of them again (IEEE 1815-2012 Table 6-1, and 4.6.6 rule 9). With the deviation off they are read once more and confirmed, and a read after that must not report them.

Source code in src/py1815/master/checks.py

def events_kept(deviation: str, what: str) -> Callable[[Bench], Plan[None]]:
    """Return the check that events are kept until the right confirmation arrives.

    The events are read with one confirmation deviation on, so the outstation
    does not receive the confirmation it asked for. Read again, it has to
    report every one of them again (IEEE 1815-2012 Table 6-1, and 4.6.6 rule
    9). With the deviation off they are read once more and confirmed, and a
    read after that must not report them.
    """

    def procedure(bench: Bench) -> Plan[None]:
        if not (yield from bench.confirming()):
            raise NotApplicable("the master is set to send no confirmations")
        yield from bench.deviate(deviation)
        first = yield from bench.events()
        # A master that polls for events has usually just taken them, so the
        # check waits for the outstation to have some, for the settling time.
        waited = 0.0
        while not _events(first) and waited < bench.settle:
            step = min(POLL_SECONDS, bench.settle - waited)
            yield from bench.pause(step)
            waited += step
            first = yield from bench.events()
        reported = _events(first)
        if not reported:
            raise NotApplicable("the outstation had no events to report")
        bench.note(f"{len(reported)} events were read and {what}")
        bench.require(
            any(fragment.control.con for fragment in first.fragments),
            "the response that carried the events did not ask for a confirmation",
        )

        missing = _missing(reported, _events((yield from bench.events())))
        bench.require(
            not missing,
            f"{len(missing)} of {len(reported)} events were not reported again though "
            f"{what}: {_said(missing)}",
        )

        yield from bench.conform()
        missing = _missing(reported, _events((yield from bench.events())))
        bench.require(
            not missing,
            f"{len(missing)} of {len(reported)} events were gone before they were "
            f"confirmed: {_said(missing)}",
        )

        # An event with no time can be equal to a later event at the same
        # point, so only the timed ones can be told to have been cleared.
        timed = [event for event in reported if event.time_ms is not None]
        if not timed:
            bench.note("the events carry no time, so their clearing could not be checked")
            return
        afterwards = {_key(each) for each in _events((yield from bench.events()))}
        repeated = [event for event in timed if _key(event) in afterwards]
        bench.require(
            not repeated,
            f"{len(repeated)} events were reported again after they were confirmed: "
            f"{_said(repeated)}",
        )

    return procedure

select ¶

select(wanted: Sequence[str] | None) -> tuple[Check, ...]

Return the checks named, in the catalog's order, or all of them for None.

A name is a check's identifier or a group, in any case. Raises ValueError for a name that is neither.

Source code in src/py1815/master/checks.py

def select(wanted: Sequence[str] | None) -> tuple[Check, ...]:
    """Return the checks named, in the catalog's order, or all of them for None.

    A name is a check's identifier or a group, in any case. Raises ValueError
    for a name that is neither.
    """
    if not wanted:
        return CATALOG
    names = {each.casefold() for each in wanted}
    known = {check.id.casefold() for check in CATALOG} | {check.group for check in CATALOG}
    unknown = sorted(names - known)
    if unknown:
        raise ValueError(f"{unknown[0]!r} is not a check; see `py1815-master evaluate --list`")
    return tuple(check for check in CATALOG if check.id.casefold() in names or check.group in names)

Deviations¶

Misbehaving on purpose is the guide.

py1815.master.deviations ¶

Named ways the master breaks a protocol rule on purpose, for testing an outstation.

An evaluation tool has to be able to do the wrong thing and say exactly what it did, so a test can see whether an outstation rejects what the standard says it must. The certification procedures already send malformed frames to a session in this process; this module gives each kind of misbehavior one name and one implementation, usable over a socket against a device on a bench as well.

A deviation is applied to the octets the association produced, on their way to the channel. The association's own code has no branch that does the wrong thing, so the conformant path cannot be bent by a flag left on: with every deviation off, :meth:Deviations.outbound returns what it was given, unchanged. Each is off until it is turned on, and turning one on changes only what the master sends, never what it expects back.

The frame-level deviations work on whole link frames, so they are built and corrupted here rather than reaching into the association or the link layer. Nothing here does I/O: a transform takes octets and returns octets, and is tested against frames written out by hand.

Deviations `dataclass` ¶

The deviations a master has turned on, and the transform they make.

Off by default. :meth:outbound is called with the octets the association produced and returns the writes to make in their place: the same octets when nothing is on, a corrupted frame, the request twice, or nothing at all.

Source code in src/py1815/master/deviations.py

@dataclass
class Deviations:
    """The deviations a master has turned on, and the transform they make.

    Off by default. :meth:`outbound` is called with the octets the association
    produced and returns the writes to make in their place: the same octets
    when nothing is on, a corrupted frame, the request twice, or nothing at
    all.
    """

    #: Send nothing at all, in either direction.
    silence: bool = False
    #: Send each request twice, octet for octet.
    repeat_request: bool = False
    #: Send every frame from another master's address.
    second_master_address: bool = False
    #: Flip the header checksum of the first frame of each request.
    corrupt_header_crc: bool = False
    #: Flip the last data-block checksum of the first frame of each request.
    corrupt_body_crc: bool = False
    #: Drop the last octet of the first frame of each request.
    truncate: bool = False
    #: Claim a length larger than the first frame of each request carries.
    overstate_length: bool = False
    #: Send the first segment of each request with a transport sequence
    #: number that is out of order.
    break_transport_sequence: bool = False
    #: Send the first segment of each request as a continuation, so the
    #: request has no first segment.
    orphan_segment: bool = False
    #: Send the last segment of each request with its final bit clear, so
    #: the series never ends.
    endless_series: bool = False
    #: Withhold the confirmation of every solicited response.
    withhold_confirmation: bool = False
    #: Send the confirmation of each solicited response after the master's
    #: next request, and not when the response arrives.
    late_confirmation: bool = False
    #: Confirm a solicited response with a sequence number the outstation did
    #: not send.
    wrong_confirmation_sequence: bool = False
    #: Withhold the confirmation of every unsolicited response.
    withhold_unsolicited_confirmation: bool = False

    #: Confirmations held back by ``late_confirmation`` until the next request.
    _held: deque[bytes] = field(
        default_factory=lambda: deque(maxlen=HELD_CONFIRMATIONS),
        init=False,
        repr=False,
        compare=False,
    )

    #: Deviations that act on a request the master initiated.
    _ON_REQUEST = (
        "corrupt_header_crc",
        "corrupt_body_crc",
        "truncate",
        "overstate_length",
        "break_transport_sequence",
        "orphan_segment",
        "endless_series",
    )
    #: Deviations that act on a confirmation the master sends in reply.
    _ON_CONFIRM = (
        "withhold_confirmation",
        "late_confirmation",
        "wrong_confirmation_sequence",
        "withhold_unsolicited_confirmation",
    )

    @classmethod
    def names(cls) -> tuple[str, ...]:
        """Every deviation's name, in the order they are listed."""
        return (
            "silence",
            "repeat_request",
            "second_master_address",
            *cls._ON_REQUEST,
            *cls._ON_CONFIRM,
        )

    def any_on(self) -> bool:
        """Whether any deviation is on."""
        return any(getattr(self, name) for name in self.names())

    def on(self) -> tuple[str, ...]:
        """The names of the deviations that are on, in order."""
        return tuple(name for name in self.names() if getattr(self, name))

    def set(self, name: str, state: bool) -> None:
        """Turn a deviation on or off by name. Raises ``ValueError`` for an unknown name."""
        if name not in self.names():
            raise ValueError(f"{name!r} is not a deviation; one of {', '.join(self.names())}")
        setattr(self, name, bool(state))

    def clear(self) -> None:
        """Turn every deviation off, and forget any confirmation held back."""
        for name in self.names():
            setattr(self, name, False)
        self._held.clear()

    def describe(self) -> dict[str, bool]:
        """Every deviation and whether it is on, as a JSON-compatible dict."""
        return {name: bool(getattr(self, name)) for name in self.names()}

    def outbound(self, octets: bytes, *, kind: str) -> list[bytes]:
        """Return the writes to make in place of ``octets``.

        ``kind`` is :data:`REQUEST` or :data:`CONFIRM`. The list is empty when
        the master is to send nothing, and is ``[octets]`` with every
        deviation off. A deviation that changes a frame returns each link
        frame as a write of its own, so a trace can keep the changed frame
        apart from the others. A request deviation changes the first frame,
        except ``endless_series``, which changes the last, and
        ``second_master_address``, which changes every frame. A confirmation
        deviation acts on each frame that carries an application confirmation
        of the kind it names, solicited or unsolicited, and leaves the other
        kind and a link-layer reply as built. A repeated request holds its
        writes twice. A confirmation held back by ``late_confirmation``
        follows the next request.
        """
        if kind not in KINDS:
            raise ValueError(f"{kind!r} is not an outbound kind; one of {', '.join(KINDS)}")
        if not self.late_confirmation:
            # A confirmation held while the deviation was on is not sent once it is off.
            self._held.clear()
        if self.silence:
            return []
        if kind == CONFIRM:
            return self._confirmation(octets)
        changes = [
            (transform, which)
            for name, transform, which in _REQUEST_TRANSFORMS
            if getattr(self, name)
        ]
        writes = self._change(octets, changes) if changes else [octets]
        if self.repeat_request:
            writes = writes * 2
        if self._held:
            writes = writes + list(self._held)
            self._held.clear()
        return writes

    def _confirmation(self, octets: bytes) -> list[bytes]:
        """Return the writes to make in place of what the association sends in reply."""
        deviating = (
            self.withhold_confirmation
            or self.late_confirmation
            or self.wrong_confirmation_sequence
            or self.withhold_unsolicited_confirmation
            or self.second_master_address
        )
        if not deviating:
            return [octets]
        writes = []
        for frame in _split(octets):
            if self.second_master_address:
                frame = second_master_address(frame)
            confirmed = _confirms(frame)
            if confirmed is None:
                # A link-layer reply sent with the confirmation is not a
                # confirmation, and is sent as built.
                writes.append(frame)
                continue
            if confirmed == _UNSOLICITED:
                # The other confirmation deviations are about a response the
                # master asked for, and leave this confirmation as built.
                if not self.withhold_unsolicited_confirmation:
                    writes.append(frame)
                continue
            if self.withhold_confirmation:
                continue
            if self.wrong_confirmation_sequence:
                frame = wrong_application_sequence(frame)
            if self.late_confirmation:
                self._held.append(frame)
            else:
                writes.append(frame)
        return writes

    @staticmethod
    def _change(octets: bytes, changes: list[tuple[Callable[[bytes], bytes], str]]) -> list[bytes]:
        """Return the link frames of ``octets``, changed by each transform in turn.

        A request is almost always one frame. A long request is several, and
        a deviation changes the first, which is where a transport or header
        fault is seen first, unless it is one that names the last frame or
        every frame.
        """
        frames = _split(octets)
        for transform, which in changes:
            if which == _EVERY:
                frames = [transform(frame) for frame in frames]
            else:
                position = 0 if which == _FIRST else -1
                frames[position] = transform(frames[position])
        return frames

names `classmethod` ¶

names() -> tuple[str, ...]

Every deviation's name, in the order they are listed.

Source code in src/py1815/master/deviations.py

@classmethod
def names(cls) -> tuple[str, ...]:
    """Every deviation's name, in the order they are listed."""
    return (
        "silence",
        "repeat_request",
        "second_master_address",
        *cls._ON_REQUEST,
        *cls._ON_CONFIRM,
    )

any_on ¶

any_on() -> bool

Whether any deviation is on.

Source code in src/py1815/master/deviations.py

def any_on(self) -> bool:
    """Whether any deviation is on."""
    return any(getattr(self, name) for name in self.names())

on ¶

on() -> tuple[str, ...]

The names of the deviations that are on, in order.

Source code in src/py1815/master/deviations.py

def on(self) -> tuple[str, ...]:
    """The names of the deviations that are on, in order."""
    return tuple(name for name in self.names() if getattr(self, name))

set ¶

set(name: str, state: bool) -> None

Turn a deviation on or off by name. Raises ValueError for an unknown name.

Source code in src/py1815/master/deviations.py

def set(self, name: str, state: bool) -> None:
    """Turn a deviation on or off by name. Raises ``ValueError`` for an unknown name."""
    if name not in self.names():
        raise ValueError(f"{name!r} is not a deviation; one of {', '.join(self.names())}")
    setattr(self, name, bool(state))

clear ¶

clear() -> None

Turn every deviation off, and forget any confirmation held back.

Source code in src/py1815/master/deviations.py

def clear(self) -> None:
    """Turn every deviation off, and forget any confirmation held back."""
    for name in self.names():
        setattr(self, name, False)
    self._held.clear()

describe ¶

describe() -> dict[str, bool]

Every deviation and whether it is on, as a JSON-compatible dict.

Source code in src/py1815/master/deviations.py

def describe(self) -> dict[str, bool]:
    """Every deviation and whether it is on, as a JSON-compatible dict."""
    return {name: bool(getattr(self, name)) for name in self.names()}

outbound ¶

outbound(octets: bytes, *, kind: str) -> list[bytes]

Return the writes to make in place of octets.

kind is :data:REQUEST or :data:CONFIRM. The list is empty when the master is to send nothing, and is [octets] with every deviation off. A deviation that changes a frame returns each link frame as a write of its own, so a trace can keep the changed frame apart from the others. A request deviation changes the first frame, except endless_series, which changes the last, and second_master_address, which changes every frame. A confirmation deviation acts on each frame that carries an application confirmation of the kind it names, solicited or unsolicited, and leaves the other kind and a link-layer reply as built. A repeated request holds its writes twice. A confirmation held back by late_confirmation follows the next request.

Source code in src/py1815/master/deviations.py

def outbound(self, octets: bytes, *, kind: str) -> list[bytes]:
    """Return the writes to make in place of ``octets``.

    ``kind`` is :data:`REQUEST` or :data:`CONFIRM`. The list is empty when
    the master is to send nothing, and is ``[octets]`` with every
    deviation off. A deviation that changes a frame returns each link
    frame as a write of its own, so a trace can keep the changed frame
    apart from the others. A request deviation changes the first frame,
    except ``endless_series``, which changes the last, and
    ``second_master_address``, which changes every frame. A confirmation
    deviation acts on each frame that carries an application confirmation
    of the kind it names, solicited or unsolicited, and leaves the other
    kind and a link-layer reply as built. A repeated request holds its
    writes twice. A confirmation held back by ``late_confirmation``
    follows the next request.
    """
    if kind not in KINDS:
        raise ValueError(f"{kind!r} is not an outbound kind; one of {', '.join(KINDS)}")
    if not self.late_confirmation:
        # A confirmation held while the deviation was on is not sent once it is off.
        self._held.clear()
    if self.silence:
        return []
    if kind == CONFIRM:
        return self._confirmation(octets)
    changes = [
        (transform, which)
        for name, transform, which in _REQUEST_TRANSFORMS
        if getattr(self, name)
    ]
    writes = self._change(octets, changes) if changes else [octets]
    if self.repeat_request:
        writes = writes * 2
    if self._held:
        writes = writes + list(self._held)
        self._held.clear()
    return writes

corrupt_header_crc ¶

corrupt_header_crc(frame: bytes) -> bytes

Flip the frame's header checksum, so the outstation cannot trust the header.

Source code in src/py1815/master/deviations.py

def corrupt_header_crc(frame: bytes) -> bytes:
    """Flip the frame's header checksum, so the outstation cannot trust the header."""
    return frame[:8] + bytes([frame[8] ^ 0xFF]) + frame[9:]

corrupt_body_crc ¶

corrupt_body_crc(frame: bytes) -> bytes

Flip the checksum of the frame's last data block.

Source code in src/py1815/master/deviations.py

def corrupt_body_crc(frame: bytes) -> bytes:
    """Flip the checksum of the frame's last data block."""
    return frame[:-1] + bytes([frame[-1] ^ 0xFF])

truncate ¶

truncate(frame: bytes) -> bytes

Drop the frame's last octet, so it is shorter than its length octet says.

Source code in src/py1815/master/deviations.py

def truncate(frame: bytes) -> bytes:
    """Drop the frame's last octet, so it is shorter than its length octet says."""
    return frame[:-1] if frame else frame

overstate_length ¶

overstate_length(frame: bytes) -> bytes

Make the length octet describe more user data than the frame carries.

The length octet is raised and the header checksum recomputed, so the outstation reads a length its header checksum vouches for, then finds fewer octets than it was promised. A length octet already at its highest value cannot be raised, so that frame loses its last data block and keeps its header as built.

Source code in src/py1815/master/deviations.py

def overstate_length(frame: bytes) -> bytes:
    """Make the length octet describe more user data than the frame carries.

    The length octet is raised and the header checksum recomputed, so the
    outstation reads a length its header checksum vouches for, then finds
    fewer octets than it was promised. A length octet already at its highest
    value cannot be raised, so that frame loses its last data block and keeps
    its header as built.
    """
    if len(frame) < link.HEADER_SIZE:
        return frame
    if frame[2] >= _MAX_LENGTH:
        block = link.DATA_BLOCK_SIZE + crc.SIZE
        return frame[: -((len(frame) - link.HEADER_SIZE) % block or block)]
    header = bytearray(frame[:8])
    header[2] = min(_MAX_LENGTH, header[2] + 16)
    return crc.encode(bytes(header)) + frame[link.HEADER_SIZE :]

break_transport_sequence ¶

break_transport_sequence(frame: bytes) -> bytes

Change the transport sequence number, so the segment lands out of order.

Source code in src/py1815/master/deviations.py

def break_transport_sequence(frame: bytes) -> bytes:
    """Change the transport sequence number, so the segment lands out of order."""
    parsed = link.parse(frame)
    if not parsed.payload:
        return frame
    header = parsed.payload[0]
    sequence = (header + 1) % transport.SEQUENCE_MODULUS
    changed = (header & ~transport.SEQ_MASK) | sequence
    return _rebuild(frame, bytes([changed]) + parsed.payload[1:])

orphan_segment ¶

orphan_segment(frame: bytes) -> bytes

Clear the first-segment bit, so a lone segment looks like a continuation.

Source code in src/py1815/master/deviations.py

def orphan_segment(frame: bytes) -> bytes:
    """Clear the first-segment bit, so a lone segment looks like a continuation."""
    parsed = link.parse(frame)
    if not parsed.payload:
        return frame
    return _rebuild(frame, bytes([parsed.payload[0] & ~transport.FIR_MASK]) + parsed.payload[1:])

endless_series ¶

endless_series(frame: bytes) -> bytes

Clear the final-segment bit, so the series the segment belongs to never ends.

The outstation is left assembling a fragment whose last segment does not come. A new first segment ends that (IEEE 1815-2012 8.2.4, Table 8-1).

Source code in src/py1815/master/deviations.py

def endless_series(frame: bytes) -> bytes:
    """Clear the final-segment bit, so the series the segment belongs to never ends.

    The outstation is left assembling a fragment whose last segment does not
    come. A new first segment ends that (IEEE 1815-2012 8.2.4, Table 8-1).
    """
    parsed = link.parse(frame)
    if not parsed.payload:
        return frame
    return _rebuild(frame, bytes([parsed.payload[0] & ~transport.FIN_MASK]) + parsed.payload[1:])

second_master_address ¶

second_master_address(frame: bytes) -> bytes

Change the frame's source address to another master's, header checksum recomputed.

The address is the next one above the master's own that is a station address and is not the outstation's. Whether an outstation answers a master it was not told of is its own choice, which its Device Profile states, so a response to this is addressed to that other master.

Source code in src/py1815/master/deviations.py

def second_master_address(frame: bytes) -> bytes:
    """Change the frame's source address to another master's, header checksum recomputed.

    The address is the next one above the master's own that is a station
    address and is not the outstation's. Whether an outstation answers a
    master it was not told of is its own choice, which its Device Profile
    states, so a response to this is addressed to that other master.
    """
    if len(frame) < link.HEADER_SIZE:
        return frame
    destination = int.from_bytes(frame[4:6], "little")
    source = int.from_bytes(frame[6:8], "little")
    other = next(
        candidate
        for candidate in (source + 1, source + 2, source - 1, source - 2)
        if link.is_valid_address(candidate) and candidate != destination
    )
    return crc.encode(frame[:6] + other.to_bytes(2, "little")) + frame[link.HEADER_SIZE :]

wrong_application_sequence ¶

wrong_application_sequence(frame: bytes) -> bytes

Change the application sequence number of the fragment the frame carries.

For a confirmation, this is confirming a sequence the outstation did not send, which must not retire the events it was waiting to have confirmed.

Source code in src/py1815/master/deviations.py

def wrong_application_sequence(frame: bytes) -> bytes:
    """Change the application sequence number of the fragment the frame carries.

    For a confirmation, this is confirming a sequence the outstation did not
    send, which must not retire the events it was waiting to have confirmed.
    """
    parsed = link.parse(frame)
    if len(parsed.payload) < 2:
        return frame
    control = parsed.payload[1]
    sequence = (control + 1) % 16
    changed = (control & 0xF0) | sequence
    return _rebuild(frame, parsed.payload[:1] + bytes([changed]) + parsed.payload[2:])

What is done unasked¶

py1815.master.tasks ¶

Automatic master tasks: the requests a master sends without being asked.

A conformant master runs a startup sequence when it connects, clears the restart indication, writes the time when the outstation needs it, and polls for events when a response indicates some are waiting. Each of these is a task that can be disabled.

:class:Housekeeper decides which requests to send. It is given the internal indications (IIN) of each response and returns the next request. It does no I/O, so the connection owner sends the requests and the logic can be tested without a socket.

Loop prevention: a task is never triggered without bound by the responses to its own requests.

Restart and need-time are edge-triggered. They fire when the bit appears and not again until it has cleared and reappeared. The integrity poll that follows a restart is made once for each appearance, including when the restart was first reported by an integrity poll.
An overflow poll fires on any response that carries the bit, except the response to the integrity poll itself.
An event poll fires on any response that says events are waiting. When the response to a poll still says so, the outstation held some back, and they are fetched at once by another event poll, up to :attr:Tasks.event_follow_ups in a row. The count starts again only when a poll's response says nothing is waiting.

An outstation that never clears a bit therefore costs a bounded number of extra requests per response, and the follow-up polls are spent once until it does.

No task operates an output.

Tasks `dataclass` ¶

Which automatic tasks are enabled.

Source code in src/py1815/master/tasks.py

@dataclass(frozen=True)
class Tasks:
    """Which automatic tasks are enabled."""

    #: Run the startup sequence on connect and when the outstation reports a
    #: restart: an integrity poll, preceded on connect by a request to disable
    #: unsolicited reporting when ``enable_unsolicited`` names a class.
    startup: bool = True
    #: Clear the restart bit (IIN1.7) when a response sets it.
    clear_restart: bool = True
    #: Write the time when a response sets need-time (IIN1.4).
    write_time: bool = True
    #: Event classes to enable unsolicited reporting for after startup.
    #: Empty means the master does not use unsolicited reporting with the
    #: outstation: it sends no request to enable it, and none to disable it.
    enable_unsolicited: tuple[int, ...] = ()
    #: Poll for events when a response sets a class 1, 2 or 3 events bit.
    events_when_indicated: bool = True
    #: Run an integrity poll when a response sets event buffer overflow
    #: (IIN2.3), because events were lost.
    integrity_on_overflow: bool = True
    #: How ``write_time`` sets the clock: None writes the master's time as it
    #: stands, and ``lan`` and ``non_lan`` follow the procedures of IEEE
    #: 1815-2012 10.3.3, which correct for the time the request takes.
    time_procedure: str | None = None
    #: Event polls made in a row when a poll's own response still says
    #: events are waiting. Zero leaves them for the next response that says so.
    event_follow_ups: int = DEFAULT_EVENT_FOLLOW_UPS

    def __post_init__(self) -> None:
        classes = tuple(self.enable_unsolicited)
        for number in classes:
            # Checked by type as well as value: int() would turn 1.9, "1" and
            # True into class 1 instead of rejecting them.
            if isinstance(number, bool) or not isinstance(number, int) or number not in (1, 2, 3):
                raise ValueError("unsolicited reporting is by event class: 1, 2 or 3")
        object.__setattr__(self, "enable_unsolicited", classes)
        if self.time_procedure is not None and self.time_procedure not in TIME_PROCEDURES:
            raise ValueError("time_procedure is null, lan or non_lan")
        follow_ups = self.event_follow_ups
        if isinstance(follow_ups, bool) or not isinstance(follow_ups, int) or follow_ups < 0:
            raise ValueError("event_follow_ups is a count of polls, zero or more")

    @classmethod
    def none(cls) -> Tasks:
        """Return a configuration with every task disabled."""
        return cls(
            startup=False,
            clear_restart=False,
            write_time=False,
            events_when_indicated=False,
            integrity_on_overflow=False,
            event_follow_ups=0,
        )

    def changed(self, given: Mapping[str, Any]) -> Tasks:
        """Return a copy with the settings in ``given`` applied.

        Raises ``ValueError`` for an unknown task name or a value of the wrong
        type.
        """
        known = {each.name for each in fields(self)}
        unknown = sorted(set(given) - known)
        if unknown:
            raise ValueError(f"{unknown[0]!r} is not a task; one of {', '.join(sorted(known))}")
        chosen: dict[str, Any] = {}
        for name, value in given.items():
            if name == "enable_unsolicited":
                if isinstance(value, (str, bytes)) or not hasattr(value, "__iter__"):
                    raise ValueError("enable_unsolicited is a list of event classes")
                chosen[name] = tuple(value)
            elif name in ("time_procedure", "event_follow_ups"):
                # Checked when the copy is made.
                chosen[name] = value
            elif not isinstance(value, bool):
                raise ValueError(f"{name} is true or false")
            else:
                chosen[name] = value
        return replace(self, **chosen)

    def reading_only(self) -> Tasks:
        """Return a copy with the tasks that write to the outstation disabled."""
        return replace(self, clear_restart=False, write_time=False)

    def describe(self) -> dict[str, Any]:
        """Return the settings as a JSON-compatible dict."""
        return {
            each.name: (
                list(self.enable_unsolicited)
                if each.name == "enable_unsolicited"
                else getattr(self, each.name)
            )
            for each in fields(self)
        }

none `classmethod` ¶

none() -> Tasks

Return a configuration with every task disabled.

Source code in src/py1815/master/tasks.py

@classmethod
def none(cls) -> Tasks:
    """Return a configuration with every task disabled."""
    return cls(
        startup=False,
        clear_restart=False,
        write_time=False,
        events_when_indicated=False,
        integrity_on_overflow=False,
        event_follow_ups=0,
    )

changed ¶

changed(given: Mapping[str, Any]) -> Tasks

Return a copy with the settings in given applied.

Raises ValueError for an unknown task name or a value of the wrong type.

Source code in src/py1815/master/tasks.py

def changed(self, given: Mapping[str, Any]) -> Tasks:
    """Return a copy with the settings in ``given`` applied.

    Raises ``ValueError`` for an unknown task name or a value of the wrong
    type.
    """
    known = {each.name for each in fields(self)}
    unknown = sorted(set(given) - known)
    if unknown:
        raise ValueError(f"{unknown[0]!r} is not a task; one of {', '.join(sorted(known))}")
    chosen: dict[str, Any] = {}
    for name, value in given.items():
        if name == "enable_unsolicited":
            if isinstance(value, (str, bytes)) or not hasattr(value, "__iter__"):
                raise ValueError("enable_unsolicited is a list of event classes")
            chosen[name] = tuple(value)
        elif name in ("time_procedure", "event_follow_ups"):
            # Checked when the copy is made.
            chosen[name] = value
        elif not isinstance(value, bool):
            raise ValueError(f"{name} is true or false")
        else:
            chosen[name] = value
    return replace(self, **chosen)

reading_only ¶

reading_only() -> Tasks

Return a copy with the tasks that write to the outstation disabled.

Source code in src/py1815/master/tasks.py

def reading_only(self) -> Tasks:
    """Return a copy with the tasks that write to the outstation disabled."""
    return replace(self, clear_restart=False, write_time=False)

describe ¶

describe() -> dict[str, Any]

Return the settings as a JSON-compatible dict.

Source code in src/py1815/master/tasks.py

def describe(self) -> dict[str, Any]:
    """Return the settings as a JSON-compatible dict."""
    return {
        each.name: (
            list(self.enable_unsolicited)
            if each.name == "enable_unsolicited"
            else getattr(self, each.name)
        )
        for each in fields(self)
    }

Step ¶

Bases: NamedTuple

One request to send, and the task it belongs to.

A step with a plan is a time synchronization of two requests: function and body are its first, and the owner carries out the plan instead.

Source code in src/py1815/master/tasks.py

class Step(NamedTuple):
    """One request to send, and the task it belongs to.

    A step with a ``plan`` is a time synchronization of two requests:
    ``function`` and ``body`` are its first, and the owner carries out the
    plan instead.
    """

    task: str
    function: FunctionCode
    body: bytes
    plan: TimeSync | None = None

Housekeeper ¶

Decide which automatic requests to send, based on response indications.

Call :meth:connected when a connection is made, :meth:saw with the indications of every response, and :meth:next to get each request that is due.

Source code in src/py1815/master/tasks.py

class Housekeeper:
    """Decide which automatic requests to send, based on response indications.

    Call :meth:`connected` when a connection is made, :meth:`saw` with the
    indications of every response, and :meth:`next` to get each request
    that is due.
    """

    def __init__(
        self, tasks: Tasks | None = None, *, clock_ms: Callable[[], int] = _wall_clock_ms
    ) -> None:
        self._tasks = Tasks() if tasks is None else tasks
        self._clock_ms = clock_ms
        self._due: set[str] = set()
        #: True from the start of a startup sequence until its integrity poll
        #: ends. A restart bit seen during that time does not start another.
        self._starting = False
        #: Edge detection: True while the bit is set and already handled.
        self._restart_seen = False
        self._time_asked = False
        #: Event polls queued in a row because a poll's own response still
        #: said events were waiting.
        self._follow_ups = 0

    @property
    def tasks(self) -> Tasks:
        """The task settings."""
        return self._tasks

    @tasks.setter
    def tasks(self, tasks: Tasks) -> None:
        """Change the settings. A due task that is now off is not done.

        A startup sequence under way finishes, and an integrity poll already
        due is still made, whatever the new settings say: either may have been
        queued for more than one reason. A request to disable unsolicited
        reporting is part of startup only for a master set to use it, so one
        that is due is dropped when the classes to enable are taken away.
        """
        self._tasks = tasks
        off = {
            "clear_restart": not tasks.clear_restart,
            "write_time": not tasks.write_time,
            "disable_unsolicited": not tasks.enable_unsolicited,
            "enable_unsolicited": not tasks.enable_unsolicited,
            "events": not tasks.events_when_indicated,
        }
        self._due -= {name for name, dropped in off.items() if dropped}

    @property
    def due(self) -> tuple[str, ...]:
        """The pending tasks, in execution order."""
        return tuple(name for name in ORDER if name in self._due)

    def connected(self) -> None:
        """Reset state for a new connection and queue the startup sequence."""
        self._due.clear()
        self._restart_seen = self._time_asked = False
        self._starting = False
        self._follow_ups = 0
        if self.tasks.startup:
            self._begin_startup(connecting=True)
        elif self.tasks.enable_unsolicited:
            self._due.add("enable_unsolicited")

    def _begin_startup(self, *, connecting: bool) -> None:
        """Queue the startup sequence, for a new connection or a reported restart.

        A master set to use unsolicited reporting with the outstation first
        asks it to stop, when the master is the one starting (IEEE 1815-2012
        5.1.1.2, rule 1). A master not set to use it sends no such request: an
        outstation without the feature need not support the function. An
        outstation that reports a restart has disabled unsolicited reporting
        itself, and the sequence for that has no request to disable it
        (5.1.1.1.2, rule 2).
        """
        self._starting = True
        self._due.add("integrity")
        if self.tasks.enable_unsolicited:
            if connecting:
                self._due.add("disable_unsolicited")
            self._due.add("enable_unsolicited")

    def saw(
        self, indications: IIN | None, *, answering: tuple[FunctionCode, bytes] | None = None
    ) -> None:
        """Queue the tasks that a response's indications call for.

        Args:
            indications: The response's IIN, or None if no response arrived.
            answering: The request that was answered, as (function code,
                request body), or None for an unsolicited response. Used to
                avoid re-triggering a poll from its own response.
        """
        integrity = answering == (FunctionCode.READ, requests.scan("integrity"))
        polled = integrity or answering == (FunctionCode.READ, requests.scan("events"))
        # Startup ends with its integrity poll, whether or not it was answered.
        # A restart bit seen after that is a new restart.
        starting, self._starting = self._starting, self._starting and not integrity
        if indications is None:
            return
        tasks = self.tasks

        if indications.is_set(IINBit.DEVICE_RESTART):
            if not self._restart_seen:
                self._restart_seen = True
                if tasks.clear_restart:
                    self._due.add("clear_restart")
                # A restart seen while startup is under way is the one it is
                # settling, with one exception. Seen first in the answer to
                # startup's own integrity poll, it was not cleared before
                # that poll, and the poll comes after the clearing (IEEE
                # 1815-2012 5.1.1.1.2, rule 2). So the poll is made once more,
                # by a master that clears the indication. This is once for
                # each time the indication appears, so it cannot repeat.
                again = integrity and tasks.clear_restart
                if tasks.startup and (not starting or again):
                    self._begin_startup(connecting=False)
        else:
            self._restart_seen = False

        if indications.is_set(IINBit.NEED_TIME):
            if not self._time_asked:
                self._time_asked = True
                if tasks.write_time:
                    self._due.add("write_time")
        else:
            self._time_asked = False

        # Overflow means events were lost, so re-read every value. Skip this
        # for the integrity poll's own response, which still carries the bit
        # until its events are confirmed.
        lost = indications.is_set(IIN2Bit.EVENT_BUFFER_OVERFLOW)
        if lost and tasks.integrity_on_overflow and not integrity:
            self._due.add("integrity")

        waiting = any(indications.is_set(bit) for bit in _EVENT_BITS)
        wanted = waiting and tasks.events_when_indicated and "integrity" not in self._due
        if not polled:
            if wanted:
                self._due.add("events")
        elif not waiting:
            self._follow_ups = 0
        elif wanted and self._follow_ups < tasks.event_follow_ups:
            # The poll's own response says more are waiting: the outstation
            # held some back. Fetched now, a bounded number of times in a row.
            self._follow_ups += 1
            self._due.add("events")

    def next(self) -> Step | None:
        """Return the next request to send, or None when nothing is due."""
        for name in ORDER:
            if name in self._due:
                self._due.discard(name)
                return self._step(name)
        return None

    def _step(self, name: str) -> Step:
        if name == "disable_unsolicited":
            return Step(name, FunctionCode.DISABLE_UNSOLICITED, requests.class_scan(1, 2, 3))
        if name == "clear_restart":
            return Step(name, FunctionCode.WRITE, requests.clear_restart())
        if name == "write_time":
            procedure = self.tasks.time_procedure
            if procedure is None:
                return Step(name, FunctionCode.WRITE, requests.write_time(self._clock_ms()))
            first = (
                FunctionCode.RECORD_CURRENT_TIME if procedure == LAN else FunctionCode.DELAY_MEASURE
            )
            return Step(name, first, b"", plan=TimeSync(procedure, clock_ms=self._clock_ms))
        if name == "integrity":
            # An integrity poll also reads events, so drop a pending event poll.
            self._due.discard("events")
            return Step(name, FunctionCode.READ, requests.scan("integrity"))
        if name == "events":
            return Step(name, FunctionCode.READ, requests.scan("events"))
        return Step(
            name,
            FunctionCode.ENABLE_UNSOLICITED,
            requests.class_scan(*self.tasks.enable_unsolicited),
        )

tasks `property` `writable` ¶

tasks: Tasks

The task settings.

due `property` ¶

due: tuple[str, ...]

The pending tasks, in execution order.

connected ¶

connected() -> None

Reset state for a new connection and queue the startup sequence.

Source code in src/py1815/master/tasks.py

def connected(self) -> None:
    """Reset state for a new connection and queue the startup sequence."""
    self._due.clear()
    self._restart_seen = self._time_asked = False
    self._starting = False
    self._follow_ups = 0
    if self.tasks.startup:
        self._begin_startup(connecting=True)
    elif self.tasks.enable_unsolicited:
        self._due.add("enable_unsolicited")

saw ¶

saw(
    indications: IIN | None,
    *,
    answering: tuple[FunctionCode, bytes] | None = None,
) -> None

Queue the tasks that a response's indications call for.

Parameters:

Name	Type	Description	Default
`indications`	`IIN \| None`	The response's IIN, or None if no response arrived.	required
`answering`	`tuple[FunctionCode, bytes] \| None`	The request that was answered, as (function code, request body), or None for an unsolicited response. Used to avoid re-triggering a poll from its own response.	`None`

Source code in src/py1815/master/tasks.py

def saw(
    self, indications: IIN | None, *, answering: tuple[FunctionCode, bytes] | None = None
) -> None:
    """Queue the tasks that a response's indications call for.

    Args:
        indications: The response's IIN, or None if no response arrived.
        answering: The request that was answered, as (function code,
            request body), or None for an unsolicited response. Used to
            avoid re-triggering a poll from its own response.
    """
    integrity = answering == (FunctionCode.READ, requests.scan("integrity"))
    polled = integrity or answering == (FunctionCode.READ, requests.scan("events"))
    # Startup ends with its integrity poll, whether or not it was answered.
    # A restart bit seen after that is a new restart.
    starting, self._starting = self._starting, self._starting and not integrity
    if indications is None:
        return
    tasks = self.tasks

    if indications.is_set(IINBit.DEVICE_RESTART):
        if not self._restart_seen:
            self._restart_seen = True
            if tasks.clear_restart:
                self._due.add("clear_restart")
            # A restart seen while startup is under way is the one it is
            # settling, with one exception. Seen first in the answer to
            # startup's own integrity poll, it was not cleared before
            # that poll, and the poll comes after the clearing (IEEE
            # 1815-2012 5.1.1.1.2, rule 2). So the poll is made once more,
            # by a master that clears the indication. This is once for
            # each time the indication appears, so it cannot repeat.
            again = integrity and tasks.clear_restart
            if tasks.startup and (not starting or again):
                self._begin_startup(connecting=False)
    else:
        self._restart_seen = False

    if indications.is_set(IINBit.NEED_TIME):
        if not self._time_asked:
            self._time_asked = True
            if tasks.write_time:
                self._due.add("write_time")
    else:
        self._time_asked = False

    # Overflow means events were lost, so re-read every value. Skip this
    # for the integrity poll's own response, which still carries the bit
    # until its events are confirmed.
    lost = indications.is_set(IIN2Bit.EVENT_BUFFER_OVERFLOW)
    if lost and tasks.integrity_on_overflow and not integrity:
        self._due.add("integrity")

    waiting = any(indications.is_set(bit) for bit in _EVENT_BITS)
    wanted = waiting and tasks.events_when_indicated and "integrity" not in self._due
    if not polled:
        if wanted:
            self._due.add("events")
    elif not waiting:
        self._follow_ups = 0
    elif wanted and self._follow_ups < tasks.event_follow_ups:
        # The poll's own response says more are waiting: the outstation
        # held some back. Fetched now, a bounded number of times in a row.
        self._follow_ups += 1
        self._due.add("events")

next ¶

next() -> Step | None

Return the next request to send, or None when nothing is due.

Source code in src/py1815/master/tasks.py

def next(self) -> Step | None:
    """Return the next request to send, or None when nothing is due."""
    for name in ORDER:
        if name in self._due:
            self._due.discard(name)
            return self._step(name)
    return None

In one process¶

py1815.master.loopback ¶

A master wired straight to a session, in one process.

No socket, no thread and no waiting: a request is handed to the session, what the session returns is handed to the association, and so on until neither has anything to say. With one injected clock shared by the two, a test moves time itself.

Automatic tasks are off unless tasks is given. With tasks, :meth:Loopback.start runs the startup sequence, and pending tasks run after each request and each :meth:Loopback.listen.

This is the two halves of the library talking to each other. It is the fastest way to exercise either, and it is not evidence that either reads the standard correctly, since they share the layers below them. See docs/planning/MASTER.md.

Loopback ¶

Bases: Operations[Exchange, Operated, Synchronized]

A master association and a session, each handed what the other sent.

Source code in src/py1815/master/loopback.py

class Loopback(Operations[Exchange, Operated, Synchronized]):
    """A master association and a session, each handed what the other sent."""

    def __init__(
        self,
        session: Session,
        *,
        association: MasterAssociation | None = None,
        store: Store | None = None,
        clock: Callable[[], float] = time.monotonic,
        tasks: Tasks | None = None,
        time_ms: Callable[[], int] | None = None,
    ) -> None:
        """
        Args:
            session: The outstation's session.
            association: The master's side. Built to match the session's
                addresses when not given.
            store: Where what is read is kept. A new one when not given.
            clock: The clock a value is stamped with when it is stored. Give
                the session, the association and this the same one.
            tasks: The automatic tasks to run. Defaults to none.
            time_ms: Clock for automatic time writes, in milliseconds since
                the epoch. Defaults to the wall clock.
        """
        if association is None:
            facts = session.facts
            association = MasterAssociation(
                outstation_address=facts.outstation_address,
                master_address=1 if facts.master_address is None else facts.master_address,
                clock=clock,
            )
        self.session = session
        self.association = association
        self.store = Store() if store is None else store
        self._clock = clock
        chosen = Tasks.none() if tasks is None else tasks
        self._time_ms = time_ms
        self.housekeeper = (
            Housekeeper(chosen) if time_ms is None else Housekeeper(chosen, clock_ms=time_ms)
        )
        #: Exchanges sent by automatic tasks, in order.
        self.unasked: list[Exchange] = []
        #: Protocol rules this master is breaking on purpose, off by default.
        self.deviations = Deviations()

    def _now_ms(self) -> int:
        return super()._now_ms() if self._time_ms is None else self._time_ms()

    def start(self) -> list[Exchange]:
        """Run the startup tasks, as on a new connection, and return their exchanges."""
        self.housekeeper.connected()
        return self._keep_house()

    def _exchange(
        self, function: FunctionCode, body: bytes, *, broadcast: link.Broadcast | None = None
    ) -> Exchange:
        exchange = self._one(function, body, broadcast=broadcast)
        self._keep_house()
        return exchange

    def _one(
        self,
        function: FunctionCode,
        body: bytes,
        *,
        task: str | None = None,
        broadcast: link.Broadcast | None = None,
    ) -> Exchange:
        self._pump(self.association.request(function, body, broadcast=broadcast))
        # Everything the session was going to say, it has said. An outstation
        # that is silent here is silent, and waiting would not change it,
        # though asking again might: a read with retries left is sent again.
        while again := self.association.retry(at_once=True):
            self._pump(again)
        self.association.give_up()
        exchange = self.association.take()
        assert exchange is not None
        if task is not None:
            exchange = replace(exchange, task=task)
        self.store.apply(exchange.objects, now=self._clock())
        self.housekeeper.saw(exchange.iin, answering=(function, body))
        self._collect()
        return exchange

    def _keep_house(self) -> list[Exchange]:
        made: list[Exchange] = []
        while (step := self.housekeeper.next()) is not None:
            if step.plan is not None:
                made += self._follow(step.plan, task=step.task).exchanges
            else:
                made.append(self._one(step.function, step.body, task=step.task))
        self.unasked += made
        return made

    def _follow(self, plan: Steps, *, task: str | None = None) -> Any:
        done: list[Exchange] = []
        while (step := plan.next(done)) is not None:
            done.append(self._one(*step, task=task))
        return plan.result(done)

    def _carry_out(self, plan: Plan) -> Operated:
        # Run pending tasks only after the whole plan, so nothing is sent
        # between a select and its operate.
        operated: Operated = self._follow(plan)
        self._keep_house()
        return operated

    def _synchronize(self, plan: TimeSync) -> Synchronized:
        synchronized: Synchronized = self._follow(plan)
        self._keep_house()
        return synchronized

    def listen(self) -> list[Unsolicited]:
        """Ask the session for what it would send unasked, and take it.

        What an owner on a socket does whenever the outstation speaks. Call
        it after the outstation has recorded events, or after moving the
        clock past a retry.
        """
        self._pump_back(self.session.initiate())
        received = self._collect()
        self._keep_house()
        return received

    def _pump(self, to_session: bytes, *, kind: str = deviations.REQUEST) -> None:
        """Carry octets back and forth until both sides are quiet.

        A deviation is applied to what the association produced on its way to
        the session, as it would be on its way to a socket: the first octets
        are a request or a confirmation as ``kind`` says, and every reply the
        association makes inside the loop is a confirmation.
        """
        to_session = self._deviate(to_session, kind=kind)
        for _ in range(_MAX_PASSES):
            if not to_session:
                return
            to_master = self.session.receive(to_session) + self.session.initiate()
            reply = self.association.receive(to_master) if to_master else b""
            to_session = self._deviate(reply, kind=deviations.CONFIRM)
        raise RuntimeError("the session and the association never stopped answering each other")

    def _deviate(self, octets: bytes, *, kind: str) -> bytes:
        """Apply any deviation to octets leaving the association. Empty stays empty."""
        if not octets:
            return octets
        return b"".join(self.deviations.outbound(octets, kind=kind))

    def _pump_back(self, to_master: bytes) -> None:
        if to_master:
            self._pump(self.association.receive(to_master), kind=deviations.CONFIRM)

    def _collect(self) -> list[Unsolicited]:
        received = self.association.take_unsolicited()
        for unsolicited in received:
            self.store.apply(unsolicited.objects, now=self._clock())
            self.housekeeper.saw(unsolicited.iin)
        return received

start ¶

start() -> list[Exchange]

Run the startup tasks, as on a new connection, and return their exchanges.

Source code in src/py1815/master/loopback.py

def start(self) -> list[Exchange]:
    """Run the startup tasks, as on a new connection, and return their exchanges."""
    self.housekeeper.connected()
    return self._keep_house()

listen ¶

listen() -> list[Unsolicited]

Ask the session for what it would send unasked, and take it.

What an owner on a socket does whenever the outstation speaks. Call it after the outstation has recorded events, or after moving the clock past a retry.

Source code in src/py1815/master/loopback.py

def listen(self) -> list[Unsolicited]:
    """Ask the session for what it would send unasked, and take it.

    What an owner on a socket does whenever the outstation speaks. Call
    it after the outstation has recorded events, or after moving the
    clock past a retry.
    """
    self._pump_back(self.session.initiate())
    received = self._collect()
    self._keep_house()
    return received

The association¶

One conversation with one outstation, with no I/O: octets in, octets out.

py1815.master.association ¶

One master association: octets in, octets out.

The mirror of :mod:py1815.session. It holds what is true of a conversation with one outstation, seen from the master's end: the request that is outstanding, the fragments of its response so far, the sequence numbers of the two series, and what arrived that nobody asked for. It does no I/O and starts no timer. Its owner hands it the octets that arrived and writes the octets it returns, and time is a clock that is read when a method is called.

That is what lets a test drive it against a :class:~py1815.session.Session in one process, through as much protocol time as it likes, with no socket and no sleep.

Busy ¶

Bases: RuntimeError

A request was made while another was still waiting for its response.

Misuse and not a protocol outcome: one association carries one request at a time, and its owner is what makes a second one wait.

Source code in src/py1815/master/association.py

class Busy(RuntimeError):
    """A request was made while another was still waiting for its response.

    Misuse and not a protocol outcome: one association carries one request at
    a time, and its owner is what makes a second one wait.
    """

Outcome ¶

Bases: Enum

How an exchange ended.

Source code in src/py1815/master/association.py

class Outcome(Enum):
    """How an exchange ended."""

    #: The response arrived, to its final fragment.
    COMPLETE = "complete"
    #: Nothing more arrived in time. Whatever did arrive is still in the result.
    TIMEOUT = "timeout"
    #: The request was one that takes no response, and was sent.
    SENT = "sent"
    #: The request was dropped before a response arrived: the connection
    #: was reset, or the caller cancelled.
    ABANDONED = "abandoned"

Exchange `dataclass` ¶

One request and everything that came back for it.

A timeout and an error indication are results, not exceptions: what an outstation did with a request is the thing a caller asked to know.

Source code in src/py1815/master/association.py

@dataclass(frozen=True)
class Exchange:
    """One request and everything that came back for it.

    A timeout and an error indication are results, not exceptions: what an
    outstation did with a request is the thing a caller asked to know.
    """

    function: FunctionCode
    sequence: int
    #: The request fragment, as sent.
    request: bytes
    outcome: Outcome
    #: Each response fragment, in the order received.
    fragments: tuple[Response, ...] = ()
    #: The objects of every fragment, in the order sent.
    objects: tuple[DecodedObject, ...] = ()
    #: Fragments whose objects could not all be read, each with the reason
    #: and the octets left unread.
    undecoded: tuple[Decoded, ...] = ()
    #: Seconds from the request being built to the exchange ending.
    elapsed: float = 0.0
    #: Name of the automatic task that sent this request, or None if a
    #: caller sent it.
    task: str | None = None
    #: Times the request was sent again after a timeout, under the same
    #: sequence number and with the same octets.
    retries: int = 0
    #: The broadcast address the request was sent to, or None when it was
    #: sent to the outstation's own address.
    broadcast: link.Broadcast | None = None

    @property
    def complete(self) -> bool:
        return self.outcome is Outcome.COMPLETE

    @property
    def iin(self) -> IIN | None:
        """The indications of the last fragment received, or None if none was."""
        return self.fragments[-1].iin if self.fragments else None

iin `property` ¶

iin: IIN | None

The indications of the last fragment received, or None if none was.

Unsolicited `dataclass` ¶

An unsolicited response, as received.

Source code in src/py1815/master/association.py

@dataclass(frozen=True)
class Unsolicited:
    """An unsolicited response, as received."""

    response: Response
    objects: tuple[DecodedObject, ...]
    undecoded: Decoded | None = None

    @property
    def iin(self) -> IIN:
        return self.response.iin

    @property
    def null(self) -> bool:
        """Whether it carried no objects: the announcement of a restart."""
        return not self.response.body

null `property` ¶

null: bool

Whether it carried no objects: the announcement of a restart.

MasterAssociation ¶

The master's side of one conversation with one outstation.

Driven by three calls. :meth:request returns the octets of a request to send. :meth:receive takes what arrived and returns what to send back, which is a confirmation or nothing. :meth:expire is called when the time :meth:expires_after gave has passed, and ends an exchange that is not going to finish, after :meth:retry has said whether a read is sent again instead. A finished exchange is collected with :meth:take, and unsolicited responses with :meth:take_unsolicited.

Source code in src/py1815/master/association.py

class MasterAssociation:
    """The master's side of one conversation with one outstation.

    Driven by three calls. :meth:`request` returns the octets of a request to
    send. :meth:`receive` takes what arrived and returns what to send back,
    which is a confirmation or nothing. :meth:`expire` is called when the time
    :meth:`expires_after` gave has passed, and ends an exchange that is not
    going to finish, after :meth:`retry` has said whether a read is sent again
    instead. A finished exchange is collected with :meth:`take`, and
    unsolicited responses with :meth:`take_unsolicited`.
    """

    def __init__(
        self,
        *,
        outstation_address: int = 1024,
        master_address: int = 1,
        response_timeout: float = DEFAULT_RESPONSE_TIMEOUT,
        confirm: bool = True,
        max_fragment: int = 2048,
        read_retries: int = DEFAULT_READ_RETRIES,
        clock: Callable[[], float] = time.monotonic,
    ) -> None:
        """
        Args:
            outstation_address: The link address of the outstation spoken to.
                A frame from any other source is dropped.
            master_address: This master's own link address.
            response_timeout: Seconds to wait for a response, and then for
                each further fragment of one that is not finished.
            confirm: Whether a fragment that asks to be confirmed is. Turn it
                off to drive an outstation by hand: nothing is then sent that
                the caller did not ask for, and :meth:`confirm` sends one.
            max_fragment: The largest response fragment that will be
                reassembled, in octets.
            read_retries: Times a read that times out with nothing received
                is sent again, as IEEE 1815-2012 4.3 rule 16 has it: under the
                same sequence number and with the same octets. No other
                request is ever sent again.
            clock: A monotonic clock in seconds.
        """
        for name, address in (
            ("outstation_address", outstation_address),
            ("master_address", master_address),
        ):
            if not link.is_valid_address(address):
                raise ValueError(
                    f"{name} is {address}; a device address is 0 to {link.MAX_ADDRESS}"
                )
        if outstation_address == master_address:
            raise ValueError(
                f"outstation_address and master_address are both {master_address}; "
                "the two ends of an association have different addresses"
            )
        if not response_timeout > 0:
            raise ValueError(f"response_timeout is {response_timeout}; it is a wait")
        if isinstance(read_retries, bool) or not isinstance(read_retries, int) or read_retries < 0:
            raise ValueError(f"read_retries is {read_retries!r}; it is a count, zero or more")
        #: Times a read that times out is sent again. Takes effect from the
        #: next request.
        self.read_retries = read_retries
        self._outstation_address = outstation_address
        self._master_address = master_address
        self._response_timeout = response_timeout
        self._confirm = confirm
        self._clock = clock
        self._frames = link.FrameReader()
        self._reassembler = Reassembler(max_fragment=max_fragment)
        #: The next request's application sequence number.
        self._sequence = 0
        #: The transport sequence number the next segment sent takes.
        self._transport_sequence = 0
        self._pending: _Pending | None = None
        self._finished: Exchange | None = None
        self._unsolicited: list[Unsolicited] = []
        #: The last unsolicited fragment, octet for octet, for telling one
        #: sent again from one that is new.
        self._last_unsolicited: bytes | None = None

    # ------------------------------------------------------------ properties

    @property
    def outstation_address(self) -> int:
        return self._outstation_address

    @property
    def master_address(self) -> int:
        return self._master_address

    @property
    def busy(self) -> bool:
        """Whether a request is waiting for its response."""
        return self._pending is not None

    # --------------------------------------------------------------- sending

    def request(
        self,
        function: FunctionCode,
        body: bytes = b"",
        *,
        broadcast: link.Broadcast | None = None,
    ) -> bytes:
        """Begin an exchange, and return the octets that carry its request.

        With ``broadcast`` the request goes to that broadcast address and not
        to the outstation's own. No outstation answers a broadcast (IEEE
        1815-2012 4.5.1), so the exchange ends at once as sent, whatever the
        function code.

        Raises :class:`Busy` while another request is outstanding, and while
        a finished exchange has not been collected with :meth:`take`: the
        result of one request is not overwritten by the next.
        """
        if self._pending is not None:
            raise Busy(f"{self._pending.function.name} is still waiting for its response")
        if self._finished is not None:
            raise Busy("the last exchange has not been taken")
        if broadcast is not None:
            broadcast = link.Broadcast(broadcast)
        sequence = self._sequence
        self._sequence = (sequence + 1) % SEQUENCE_MODULUS
        fragment = build_request(function, sequence=sequence, body=body)
        now = self._clock()
        if broadcast is not None or function in NO_RESPONSE_FUNCTIONS:
            self._finished = Exchange(
                function, sequence, fragment, Outcome.SENT, broadcast=broadcast
            )
        else:
            self._pending = _Pending(
                function=function,
                sequence=sequence,
                request=fragment,
                started=now,
                deadline=now + self._response_timeout,
                expected=sequence,
                retries_left=self.read_retries if function in RETRIED_FUNCTIONS else 0,
            )
        return self._send(fragment, destination=broadcast)

    @property
    def confirming(self) -> bool:
        """Whether a response that asks for a confirmation is sent one."""
        return self._confirm

    def confirm(self, response: Response) -> bytes:
        """The octets that confirm a response fragment, for an owner that
        turned automatic confirmation off."""
        return self._send(
            build_confirm(sequence=response.control.sequence, unsolicited=response.unsolicited)
        )

    def _send(self, fragment: bytes, *, destination: int | None = None) -> bytes:
        control = link.control_byte(
            from_master=True, primary=True, function=link.PrimaryFunction.UNCONFIRMED_USER_DATA
        )
        if destination is None:
            destination = self._outstation_address
        out = bytearray()
        segments = segment(fragment, first_sequence=self._transport_sequence)
        self._transport_sequence = (
            self._transport_sequence + len(segments)
        ) % TRANSPORT_SEQUENCE_MODULUS
        for tpdu in segments:
            out += link.build(
                control,
                destination=destination,
                source=self._master_address,
                payload=tpdu,
            )
        return bytes(out)

    # ------------------------------------------------------------- receiving

    def receive(self, data: bytes) -> bytes:
        """Take octets from the outstation, and return octets to send back."""
        out = bytearray()
        for frame in self._frames.feed(data):
            out += self._handle_frame(frame)
        return bytes(out)

    def _handle_frame(self, frame: link.LinkFrame) -> bytes:
        if frame.destination != self._master_address:
            logger.debug("dnp3 master: frame for %d is not ours", frame.destination)
            return b""
        if frame.source != self._outstation_address:
            logger.warning(
                "dnp3 master: dropping frame from unexpected outstation address %d", frame.source
            )
            return b""
        if frame.from_master:
            logger.warning("dnp3 master: dropping a frame marked as sent by a master")
            return b""
        if not frame.is_primary:
            # An answer to a link request. This master sends user data that
            # asks for no link acknowledgment, so there is nothing to match.
            return b""

        function = frame.function
        if function == link.PrimaryFunction.UNCONFIRMED_USER_DATA:
            return self._handle_segment(frame.payload)
        if function == link.PrimaryFunction.CONFIRMED_USER_DATA:
            # Acknowledged and taken. A frame sent again because the
            # acknowledgment was lost carries a fragment already taken, and
            # the rules for fragments below drop it: a response nothing is
            # waiting for, or an unsolicited one repeated octet for octet.
            return self._link_reply(link.SecondaryFunction.ACK) + self._handle_segment(
                frame.payload
            )
        if function in (
            link.PrimaryFunction.RESET_LINK_STATES,
            link.PrimaryFunction.TEST_LINK_STATES,
        ):
            return self._link_reply(link.SecondaryFunction.ACK)
        if function == link.PrimaryFunction.REQUEST_LINK_STATUS:
            return self._link_reply(link.SecondaryFunction.LINK_STATUS)
        return self._link_reply(link.SecondaryFunction.NOT_SUPPORTED)

    def _link_reply(self, function: link.SecondaryFunction) -> bytes:
        control = link.control_byte(from_master=True, primary=False, function=function)
        return link.build(
            control, destination=self._outstation_address, source=self._master_address
        )

    def _handle_segment(self, tpdu: bytes) -> bytes:
        try:
            fragment = self._reassembler.add(tpdu)
        except TransportError as exc:
            logger.warning("dnp3 master: transport error: %s", exc)
            return b""
        if fragment is None:
            return b""
        return self._handle_fragment(fragment)

    def _handle_fragment(self, fragment: bytes) -> bytes:
        try:
            response = parse_response(fragment)
        except ResponseError as exc:
            logger.warning("dnp3 master: fragment dropped: %s", exc)
            return b""
        if response.unsolicited:
            return self._handle_unsolicited(response, fragment)
        return self._handle_solicited(response)

    def _handle_unsolicited(self, response: Response, fragment: bytes) -> bytes:
        # An unsolicited response is one fragment, complete in itself.
        if not (response.control.fir and response.control.fin):
            logger.warning("dnp3 master: unsolicited response that is not one fragment dropped")
            return b""
        if fragment == self._last_unsolicited:
            # The same octets under the same sequence number: the outstation
            # did not see the confirmation and sent it again. Confirmed again
            # and not delivered again.
            logger.info("dnp3 master: unsolicited response repeated; confirmed again")
        else:
            self._last_unsolicited = bytes(fragment)
            decoded = decode_objects(response.body)
            self._unsolicited.append(
                Unsolicited(response, decoded.objects, None if decoded.complete else decoded)
            )
        if response.control.con and self._confirm:
            return self.confirm(response)
        return b""

    def _handle_solicited(self, response: Response) -> bytes:
        pending = self._pending
        if pending is None:
            logger.info(
                "dnp3 master: response with sequence %d and nothing outstanding; dropped",
                response.control.sequence,
            )
            return b""
        first = not pending.fragments
        if response.control.fir != first or response.control.sequence != pending.expected:
            # Not the fragment this exchange is waiting for: a late answer to
            # a request already given up on, or a series out of order. It is
            # not confirmed, since confirming it would retire events this
            # master never read.
            logger.warning(
                "dnp3 master: fragment with sequence %d%s is not the one awaited (%d%s); dropped",
                response.control.sequence,
                " first" if response.control.fir else "",
                pending.expected,
                " first" if first else "",
            )
            return b""

        decoded = decode_objects(response.body)
        pending.fragments.append(response)
        pending.objects.extend(decoded.objects)
        if not decoded.complete:
            pending.undecoded.append(decoded)
        now = self._clock()
        out = b""
        if response.control.con and self._confirm:
            out = self.confirm(response)
        if response.control.fin:
            self._finish(Outcome.COMPLETE, now)
        else:
            pending.expected = (response.control.sequence + 1) % SEQUENCE_MODULUS
            pending.deadline = now + self._response_timeout
        return out

    # ------------------------------------------------------------------ time

    def expires_after(self) -> float | None:
        """Seconds until the outstanding request times out, or None if none is."""
        if self._pending is None:
            return None
        return max(0.0, self._pending.deadline - self._clock())

    def retry(self, *, at_once: bool = False) -> bytes:
        """Send the outstanding request again if it may be, and return its octets.

        It may be when its time has passed with nothing received, it is a
        read, and it has retries left (``read_retries``). It is sent under the
        same sequence number with the same octets (IEEE 1815-2012 4.3 rule
        16), and waits a whole response timeout again. A read whose response
        has begun to arrive is not sent again, since the outstation is
        answering it.

        Return no octets when the request is not sent again; the owner then
        calls :meth:`expire`. With ``at_once`` the time is not waited for: for
        an owner that knows nothing more is coming, as :meth:`give_up` says.
        """
        pending = self._pending
        if pending is None or pending.fragments or pending.retries_left <= 0:
            return b""
        now = self._clock()
        if not at_once and now < pending.deadline:
            return b""
        pending.retries_left -= 1
        pending.retries += 1
        pending.deadline = now + self._response_timeout
        logger.info(
            "dnp3 master: %s with sequence %d timed out; sending it again",
            pending.function.name,
            pending.sequence,
        )
        return self._send(pending.request)

    def expire(self) -> bool:
        """End the outstanding exchange if its time has passed. Says whether it did."""
        pending = self._pending
        if pending is None:
            return False
        now = self._clock()
        if now < pending.deadline:
            return False
        logger.info(
            "dnp3 master: %s timed out after %d fragment(s)",
            pending.function.name,
            len(pending.fragments),
        )
        self._finish(Outcome.TIMEOUT, now)
        return True

    def give_up(self) -> bool:
        """End the outstanding exchange now, as a timeout. Says whether one was.

        For an owner that knows nothing more is coming without waiting to find
        out: one wired straight to a session in the same process, where an
        outstation that has not answered by the time the call returns is not
        going to.
        """
        if self._pending is None:
            return False
        self._finish(Outcome.TIMEOUT, self._clock())
        return True

    def abandon(self) -> bool:
        """End the pending exchange as abandoned. Return True if there was one.

        Call this when the caller cancels a request that has already been
        sent. A late response is then dropped and not confirmed. Without this
        the association stays busy and rejects every later request.
        """
        if self._pending is None:
            return False
        self._finish(Outcome.ABANDONED, self._clock())
        return True

    def _finish(self, outcome: Outcome, now: float) -> None:
        pending = self._pending
        assert pending is not None
        self._pending = None
        self._finished = Exchange(
            function=pending.function,
            sequence=pending.sequence,
            request=pending.request,
            outcome=outcome,
            fragments=tuple(pending.fragments),
            objects=tuple(pending.objects),
            undecoded=tuple(pending.undecoded),
            elapsed=now - pending.started,
            retries=pending.retries,
        )

    # --------------------------------------------------------------- results

    def take(self) -> Exchange | None:
        """The exchange that finished, once. None while it has not."""
        finished, self._finished = self._finished, None
        return finished

    def take_unsolicited(self) -> list[Unsolicited]:
        """The unsolicited responses received since this was last called."""
        received, self._unsolicited = self._unsolicited, []
        return received

    def connection_reset(self) -> None:
        """Forget what a dead connection left behind.

        Half a frame and half a fragment mean nothing to the connection that
        replaces the one they arrived on. A request that was outstanding is
        ended as abandoned: its response, if one was coming, went with the
        connection. The sequence numbers carry on, since they belong to the
        association and not to the socket.
        """
        self._frames = link.FrameReader()
        self._reassembler.reset()
        self._last_unsolicited = None
        self.abandon()

busy `property` ¶

busy: bool

Whether a request is waiting for its response.

confirming `property` ¶

confirming: bool

Whether a response that asks for a confirmation is sent one.

request ¶

request(
    function: FunctionCode,
    body: bytes = b"",
    *,
    broadcast: Broadcast | None = None,
) -> bytes

Begin an exchange, and return the octets that carry its request.

With broadcast the request goes to that broadcast address and not to the outstation's own. No outstation answers a broadcast (IEEE 1815-2012 4.5.1), so the exchange ends at once as sent, whatever the function code.

Raises :class:Busy while another request is outstanding, and while a finished exchange has not been collected with :meth:take: the result of one request is not overwritten by the next.

Source code in src/py1815/master/association.py

def request(
    self,
    function: FunctionCode,
    body: bytes = b"",
    *,
    broadcast: link.Broadcast | None = None,
) -> bytes:
    """Begin an exchange, and return the octets that carry its request.

    With ``broadcast`` the request goes to that broadcast address and not
    to the outstation's own. No outstation answers a broadcast (IEEE
    1815-2012 4.5.1), so the exchange ends at once as sent, whatever the
    function code.

    Raises :class:`Busy` while another request is outstanding, and while
    a finished exchange has not been collected with :meth:`take`: the
    result of one request is not overwritten by the next.
    """
    if self._pending is not None:
        raise Busy(f"{self._pending.function.name} is still waiting for its response")
    if self._finished is not None:
        raise Busy("the last exchange has not been taken")
    if broadcast is not None:
        broadcast = link.Broadcast(broadcast)
    sequence = self._sequence
    self._sequence = (sequence + 1) % SEQUENCE_MODULUS
    fragment = build_request(function, sequence=sequence, body=body)
    now = self._clock()
    if broadcast is not None or function in NO_RESPONSE_FUNCTIONS:
        self._finished = Exchange(
            function, sequence, fragment, Outcome.SENT, broadcast=broadcast
        )
    else:
        self._pending = _Pending(
            function=function,
            sequence=sequence,
            request=fragment,
            started=now,
            deadline=now + self._response_timeout,
            expected=sequence,
            retries_left=self.read_retries if function in RETRIED_FUNCTIONS else 0,
        )
    return self._send(fragment, destination=broadcast)

confirm ¶

confirm(response: Response) -> bytes

The octets that confirm a response fragment, for an owner that turned automatic confirmation off.

Source code in src/py1815/master/association.py

def confirm(self, response: Response) -> bytes:
    """The octets that confirm a response fragment, for an owner that
    turned automatic confirmation off."""
    return self._send(
        build_confirm(sequence=response.control.sequence, unsolicited=response.unsolicited)
    )

receive ¶

receive(data: bytes) -> bytes

Take octets from the outstation, and return octets to send back.

Source code in src/py1815/master/association.py

def receive(self, data: bytes) -> bytes:
    """Take octets from the outstation, and return octets to send back."""
    out = bytearray()
    for frame in self._frames.feed(data):
        out += self._handle_frame(frame)
    return bytes(out)

expires_after ¶

expires_after() -> float | None

Seconds until the outstanding request times out, or None if none is.

Source code in src/py1815/master/association.py

def expires_after(self) -> float | None:
    """Seconds until the outstanding request times out, or None if none is."""
    if self._pending is None:
        return None
    return max(0.0, self._pending.deadline - self._clock())

retry ¶

retry(*, at_once: bool = False) -> bytes

Send the outstanding request again if it may be, and return its octets.

It may be when its time has passed with nothing received, it is a read, and it has retries left (read_retries). It is sent under the same sequence number with the same octets (IEEE 1815-2012 4.3 rule 16), and waits a whole response timeout again. A read whose response has begun to arrive is not sent again, since the outstation is answering it.

Return no octets when the request is not sent again; the owner then calls :meth:expire. With at_once the time is not waited for: for an owner that knows nothing more is coming, as :meth:give_up says.

Source code in src/py1815/master/association.py

def retry(self, *, at_once: bool = False) -> bytes:
    """Send the outstanding request again if it may be, and return its octets.

    It may be when its time has passed with nothing received, it is a
    read, and it has retries left (``read_retries``). It is sent under the
    same sequence number with the same octets (IEEE 1815-2012 4.3 rule
    16), and waits a whole response timeout again. A read whose response
    has begun to arrive is not sent again, since the outstation is
    answering it.

    Return no octets when the request is not sent again; the owner then
    calls :meth:`expire`. With ``at_once`` the time is not waited for: for
    an owner that knows nothing more is coming, as :meth:`give_up` says.
    """
    pending = self._pending
    if pending is None or pending.fragments or pending.retries_left <= 0:
        return b""
    now = self._clock()
    if not at_once and now < pending.deadline:
        return b""
    pending.retries_left -= 1
    pending.retries += 1
    pending.deadline = now + self._response_timeout
    logger.info(
        "dnp3 master: %s with sequence %d timed out; sending it again",
        pending.function.name,
        pending.sequence,
    )
    return self._send(pending.request)

expire ¶

expire() -> bool

End the outstanding exchange if its time has passed. Says whether it did.

Source code in src/py1815/master/association.py

def expire(self) -> bool:
    """End the outstanding exchange if its time has passed. Says whether it did."""
    pending = self._pending
    if pending is None:
        return False
    now = self._clock()
    if now < pending.deadline:
        return False
    logger.info(
        "dnp3 master: %s timed out after %d fragment(s)",
        pending.function.name,
        len(pending.fragments),
    )
    self._finish(Outcome.TIMEOUT, now)
    return True

give_up ¶

give_up() -> bool

End the outstanding exchange now, as a timeout. Says whether one was.

For an owner that knows nothing more is coming without waiting to find out: one wired straight to a session in the same process, where an outstation that has not answered by the time the call returns is not going to.

Source code in src/py1815/master/association.py

def give_up(self) -> bool:
    """End the outstanding exchange now, as a timeout. Says whether one was.

    For an owner that knows nothing more is coming without waiting to find
    out: one wired straight to a session in the same process, where an
    outstation that has not answered by the time the call returns is not
    going to.
    """
    if self._pending is None:
        return False
    self._finish(Outcome.TIMEOUT, self._clock())
    return True

abandon ¶

abandon() -> bool

End the pending exchange as abandoned. Return True if there was one.

Call this when the caller cancels a request that has already been sent. A late response is then dropped and not confirmed. Without this the association stays busy and rejects every later request.

Source code in src/py1815/master/association.py

def abandon(self) -> bool:
    """End the pending exchange as abandoned. Return True if there was one.

    Call this when the caller cancels a request that has already been
    sent. A late response is then dropped and not confirmed. Without this
    the association stays busy and rejects every later request.
    """
    if self._pending is None:
        return False
    self._finish(Outcome.ABANDONED, self._clock())
    return True

take ¶

take() -> Exchange | None

The exchange that finished, once. None while it has not.

Source code in src/py1815/master/association.py

def take(self) -> Exchange | None:
    """The exchange that finished, once. None while it has not."""
    finished, self._finished = self._finished, None
    return finished

take_unsolicited ¶

take_unsolicited() -> list[Unsolicited]

The unsolicited responses received since this was last called.

Source code in src/py1815/master/association.py

def take_unsolicited(self) -> list[Unsolicited]:
    """The unsolicited responses received since this was last called."""
    received, self._unsolicited = self._unsolicited, []
    return received

connection_reset ¶

connection_reset() -> None

Forget what a dead connection left behind.

Half a frame and half a fragment mean nothing to the connection that replaces the one they arrived on. A request that was outstanding is ended as abandoned: its response, if one was coming, went with the connection. The sequence numbers carry on, since they belong to the association and not to the socket.

Source code in src/py1815/master/association.py

def connection_reset(self) -> None:
    """Forget what a dead connection left behind.

    Half a frame and half a fragment mean nothing to the connection that
    replaces the one they arrived on. A request that was outstanding is
    ended as abandoned: its response, if one was coming, went with the
    connection. The sequence numbers carry on, since they belong to the
    association and not to the socket.
    """
    self._frames = link.FrameReader()
    self._reassembler.reset()
    self._last_unsolicited = None
    self.abandon()

The store¶

py1815.master.store ¶

What a master has been told: the last value of every point, and the events.

A store is filled from the objects of responses, in the order they arrived, and is asked afterwards. It knows nothing of how the objects were obtained, so a poll, a read and an unsolicited response all feed it the same way.

PointValue `dataclass` ¶

The last thing an outstation said about one point.

Source code in src/py1815/master/store.py

@dataclass(frozen=True)
class PointValue:
    """The last thing an outstation said about one point."""

    value: bool | int | float | None
    #: The flag octet, or None when the variation that carried the value has
    #: none. None does not mean the point is healthy; it means nobody said.
    flags: int | None
    #: The time the outstation gave the value, in milliseconds since the
    #: epoch, or None when it gave none.
    time_ms: int | None
    #: Whether the value arrived as an event and not as a static object.
    from_event: bool
    group: int
    variation: int
    #: The clock when the value was received.
    received: float

Store ¶

The last value of every point an outstation has reported.

Source code in src/py1815/master/store.py

class Store:
    """The last value of every point an outstation has reported."""

    def __init__(self, *, event_capacity: int = DEFAULT_EVENT_CAPACITY) -> None:
        self._points: dict[tuple[PointType, int], PointValue] = {}
        self._events: deque[DecodedObject] = deque(maxlen=event_capacity)
        #: Events received since the store was made, including those dropped
        #: for capacity and those cleared. Counts up and never goes back.
        self.events_received = 0

    def events_since(self, received: int) -> tuple[DecodedObject, ...]:
        """Return the events received after ``events_received`` stood at ``received``.

        Only those still held: an event dropped for capacity or cleared is
        not returned.
        """
        newer = self.events_received - received
        if newer <= 0:
            return ()
        return tuple(self._events)[-newer:]

    def apply(self, objects: Iterable[DecodedObject], *, now: float) -> None:
        """Take the objects of a response, in the order they were sent.

        The order matters and is the outstation's: an integrity poll asks for
        events before static data so that a present value is not overwritten
        by an older change, and applying objects in the order sent is what
        honors that.
        """
        for decoded in objects:
            if decoded.point is None or decoded.index is None:
                continue
            self._points[(decoded.point, decoded.index)] = PointValue(
                value=decoded.value,
                flags=decoded.flags,
                time_ms=decoded.time_ms,
                from_event=decoded.event,
                group=decoded.group,
                variation=decoded.variation,
                received=now,
            )
            if decoded.event:
                self._events.append(decoded)
                self.events_received += 1

    def get(self, point: PointType, index: int) -> PointValue | None:
        """One point's last value, or None if the outstation has never sent it."""
        return self._points.get((point, index))

    def points(self, point: PointType) -> dict[int, PointValue]:
        """Every point of one type that has been reported, by index, in order."""
        found = {index: value for (kind, index), value in self._points.items() if kind is point}
        return dict(sorted(found.items()))

    def binary_input(self, index: int) -> PointValue | None:
        return self.get(PointType.BINARY_INPUT, index)

    def binary_output(self, index: int) -> PointValue | None:
        return self.get(PointType.BINARY_OUTPUT, index)

    def counter(self, index: int) -> PointValue | None:
        return self.get(PointType.COUNTER, index)

    def frozen_counter(self, index: int) -> PointValue | None:
        return self.get(PointType.FROZEN_COUNTER, index)

    def analog_input(self, index: int) -> PointValue | None:
        return self.get(PointType.ANALOG_INPUT, index)

    def analog_output(self, index: int) -> PointValue | None:
        return self.get(PointType.ANALOG_OUTPUT, index)

    @property
    def events(self) -> tuple[DecodedObject, ...]:
        """The events received, oldest first, up to the capacity."""
        return tuple(self._events)

    def clear_events(self) -> None:
        self._events.clear()

    def __len__(self) -> int:
        return len(self._points)

events `property` ¶

events: tuple[DecodedObject, ...]

The events received, oldest first, up to the capacity.

events_since ¶

events_since(received: int) -> tuple[DecodedObject, ...]

Return the events received after events_received stood at received.

Only those still held: an event dropped for capacity or cleared is not returned.

Source code in src/py1815/master/store.py

def events_since(self, received: int) -> tuple[DecodedObject, ...]:
    """Return the events received after ``events_received`` stood at ``received``.

    Only those still held: an event dropped for capacity or cleared is
    not returned.
    """
    newer = self.events_received - received
    if newer <= 0:
        return ()
    return tuple(self._events)[-newer:]

apply ¶

apply(
    objects: Iterable[DecodedObject], *, now: float
) -> None

Take the objects of a response, in the order they were sent.

The order matters and is the outstation's: an integrity poll asks for events before static data so that a present value is not overwritten by an older change, and applying objects in the order sent is what honors that.

Source code in src/py1815/master/store.py

def apply(self, objects: Iterable[DecodedObject], *, now: float) -> None:
    """Take the objects of a response, in the order they were sent.

    The order matters and is the outstation's: an integrity poll asks for
    events before static data so that a present value is not overwritten
    by an older change, and applying objects in the order sent is what
    honors that.
    """
    for decoded in objects:
        if decoded.point is None or decoded.index is None:
            continue
        self._points[(decoded.point, decoded.index)] = PointValue(
            value=decoded.value,
            flags=decoded.flags,
            time_ms=decoded.time_ms,
            from_event=decoded.event,
            group=decoded.group,
            variation=decoded.variation,
            received=now,
        )
        if decoded.event:
            self._events.append(decoded)
            self.events_received += 1

get ¶

get(point: PointType, index: int) -> PointValue | None

One point's last value, or None if the outstation has never sent it.

Source code in src/py1815/master/store.py

def get(self, point: PointType, index: int) -> PointValue | None:
    """One point's last value, or None if the outstation has never sent it."""
    return self._points.get((point, index))

points ¶

points(point: PointType) -> dict[int, PointValue]

Every point of one type that has been reported, by index, in order.

Source code in src/py1815/master/store.py

def points(self, point: PointType) -> dict[int, PointValue]:
    """Every point of one type that has been reported, by index, in order."""
    found = {index: value for (kind, index), value in self._points.items() if kind is point}
    return dict(sorted(found.items()))

The trace¶

py1815.master.trace ¶

The record of an exchange: every frame, with what it says.

A trace is what a person reads to see what two stations said to each other. Each frame that was sent or received is kept with its time and its direction, its octets, and a reading of them one layer at a time: the data link, the transport function, and, on the frame that completes a fragment, the application layer.

The reading is for people. Nothing in the master acts on it, so a frame it describes badly is a display fault and never a protocol one.

A trace also knows which connection each frame crossed and the addresses of its two ends, so it can be exported as a pcap file (:meth:Trace.capture) or written to one as it is recorded (:class:Recorder), for a dissector to read.

Entry `dataclass` ¶

One frame, as sent or received.

Source code in src/py1815/master/trace.py

@dataclass(frozen=True)
class Entry:
    """One frame, as sent or received."""

    #: Counts up from 1 for the life of the trace, and is never reused.
    id: int
    #: Seconds since the epoch.
    at: float
    #: ``"tx"`` for a frame this master sent, ``"rx"`` for one it received.
    direction: str
    octets: bytes
    #: The data link layer's reading of the frame. For octets this master sent
    #: that are not a valid frame, the header as written and a ``problem``.
    link: dict[str, Any]
    #: The transport header, when the frame carries user data.
    transport: dict[str, Any] | None
    #: The application fragment, on the frame that completed one.
    application: dict[str, Any] | None
    #: One line that says what the frame is.
    summary: str
    #: The connection the frame crossed: 1 for the trace's first, counting up
    #: with each new one, and 0 for a frame recorded before any.
    connection: int = 0

Trace ¶

Frames, in the order they crossed the wire.

Source code in src/py1815/master/trace.py

class Trace:
    """Frames, in the order they crossed the wire."""

    def __init__(
        self, *, capacity: int = DEFAULT_CAPACITY, clock: Callable[[], float] = time.time
    ) -> None:
        self._entries: deque[Entry] = deque(maxlen=capacity)
        self._clock = clock
        self._next = 1
        self._readers = {SENT: link.FrameReader(), RECEIVED: link.FrameReader()}
        self._reassemblers = {SENT: Reassembler(), RECEIVED: Reassembler()}
        self._connection = 0
        self._endpoints: dict[int, tuple[Address | None, Address | None]] = {}
        #: Called with each entry as it is recorded.
        self.listeners: list[Callable[[Entry], None]] = []

    def record(self, direction: str, octets: bytes) -> list[Entry]:
        """Take octets that were sent or received, and keep each frame in them.

        Received octets arrive split wherever the network split them, so a
        frame is recorded when its last octet has arrived and not before.
        """
        return [
            self._keep(self._entry(direction, frame))
            for frame in self._readers[direction].feed(octets)
        ]

    def record_write(self, octets: bytes) -> list[Entry]:
        """Record one write this master made, whether or not it is valid frames.

        A deviation can make the master write a frame that no reader accepts:
        a checksum that does not verify, or fewer octets than the length octet
        describes. :meth:`record` reads octets the way a receiver does, so it
        drops such a frame, and holds a short one and reads the next write as
        the rest of it. A write that is not whole valid frames is kept here as
        one entry, with the octets as written and what is wrong with them, so
        the trace and a capture made from it show what crossed the wire.
        """
        frames = link.FrameReader().feed(octets)
        if sum(link.frame_size(len(frame.payload)) for frame in frames) == len(octets):
            return [self._keep(self._entry(SENT, frame)) for frame in frames]
        problem = "the octets are not whole frames"
        try:
            link.parse(octets)
        except link.LinkFrameError as error:
            problem = str(error)
        entry_id, self._next = self._next, self._next + 1
        entry = Entry(
            id=entry_id,
            at=self._clock(),
            direction=SENT,
            octets=octets,
            link=_malformed(octets, problem),
            transport=None,
            application=None,
            summary=f"not a valid frame: {problem}",
            connection=self._connection,
        )
        return [self._keep(entry)]

    def _keep(self, entry: Entry) -> Entry:
        self._entries.append(entry)
        for listener in list(self.listeners):
            listener(entry)
        return entry

    def _entry(self, direction: str, frame: link.LinkFrame) -> Entry:
        described = _link(frame)
        transport: dict[str, Any] | None = None
        application: dict[str, Any] | None = None
        carries_data = frame.is_primary and frame.function in (
            link.PrimaryFunction.UNCONFIRMED_USER_DATA,
            link.PrimaryFunction.CONFIRMED_USER_DATA,
        )
        if carries_data and frame.payload:
            header = frame.payload[0]
            transport = {
                "fir": bool(header & FIR_MASK),
                "fin": bool(header & FIN_MASK),
                "sequence": header & SEQ_MASK,
            }
            try:
                fragment = self._reassemblers[direction].add(frame.payload)
            except TransportError as error:
                transport["problem"] = str(error)
                fragment = None
            if fragment is not None and len(fragment) >= 2:
                application = _application(fragment)
        entry_id, self._next = self._next, self._next + 1
        return Entry(
            id=entry_id,
            at=self._clock(),
            direction=direction,
            octets=link.build(frame.control, frame.destination, frame.source, frame.payload),
            link=described,
            transport=transport,
            application=application,
            summary=_summary(described, transport, application),
            connection=self._connection,
        )

    @property
    def last_id(self) -> int:
        """The id of the newest frame recorded, or 0 when none has been."""
        return self._next - 1

    def since(self, after: int = 0) -> list[Entry]:
        """Every entry kept whose id is greater than ``after``, oldest first."""
        return [entry for entry in self._entries if entry.id > after]

    def clear(self) -> None:
        """Forget the frames kept. Ids carry on from where they were."""
        self._entries.clear()

    def reset(self, *, local: Address | None = None, peer: Address | None = None) -> None:
        """Start a new connection: forget half a frame and half a fragment.

        Args:
            local: This master's address and port on the new connection.
            peer: The outstation's address and port.
        """
        self._readers = {SENT: link.FrameReader(), RECEIVED: link.FrameReader()}
        for reassembler in self._reassemblers.values():
            reassembler.reset()
        self._connection += 1
        # Keep the endpoints of the connections a kept frame crossed, and of
        # the new one, and no others. A connection that closed before any
        # frame would otherwise stay here for as long as the master runs.
        needed = {entry.connection for entry in self._entries}
        for connection in [number for number in self._endpoints if number not in needed]:
            del self._endpoints[connection]
        self._endpoints[self._connection] = (local, peer)

    def endpoints(self, connection: int) -> tuple[Address | None, Address | None]:
        """Return this master's and the outstation's address on a connection.

        Either is None where it is not known: for frames recorded before any
        connection, and for a connection whose addresses were not given.
        """
        return self._endpoints.get(connection, (None, None))

    def capture(self, *, after: int = 0, port: int = DNP3_PORT) -> bytes:
        """Return the frames kept as a pcap file.

        Args:
            after: Only frames whose id is greater.
            port: The outstation's port, for a connection whose addresses are
                not known.
        """
        capture = Capture()
        recorder = Recorder(capture, self, port=port)
        for entry in self.since(after):
            recorder.record(entry)
        recorder.close()
        return capture.pcap()

    def __len__(self) -> int:
        return len(self._entries)

last_id `property` ¶

last_id: int

The id of the newest frame recorded, or 0 when none has been.

record ¶

record(direction: str, octets: bytes) -> list[Entry]

Take octets that were sent or received, and keep each frame in them.

Received octets arrive split wherever the network split them, so a frame is recorded when its last octet has arrived and not before.

Source code in src/py1815/master/trace.py

def record(self, direction: str, octets: bytes) -> list[Entry]:
    """Take octets that were sent or received, and keep each frame in them.

    Received octets arrive split wherever the network split them, so a
    frame is recorded when its last octet has arrived and not before.
    """
    return [
        self._keep(self._entry(direction, frame))
        for frame in self._readers[direction].feed(octets)
    ]

record_write ¶

record_write(octets: bytes) -> list[Entry]

Record one write this master made, whether or not it is valid frames.

A deviation can make the master write a frame that no reader accepts: a checksum that does not verify, or fewer octets than the length octet describes. :meth:record reads octets the way a receiver does, so it drops such a frame, and holds a short one and reads the next write as the rest of it. A write that is not whole valid frames is kept here as one entry, with the octets as written and what is wrong with them, so the trace and a capture made from it show what crossed the wire.

Source code in src/py1815/master/trace.py

def record_write(self, octets: bytes) -> list[Entry]:
    """Record one write this master made, whether or not it is valid frames.

    A deviation can make the master write a frame that no reader accepts:
    a checksum that does not verify, or fewer octets than the length octet
    describes. :meth:`record` reads octets the way a receiver does, so it
    drops such a frame, and holds a short one and reads the next write as
    the rest of it. A write that is not whole valid frames is kept here as
    one entry, with the octets as written and what is wrong with them, so
    the trace and a capture made from it show what crossed the wire.
    """
    frames = link.FrameReader().feed(octets)
    if sum(link.frame_size(len(frame.payload)) for frame in frames) == len(octets):
        return [self._keep(self._entry(SENT, frame)) for frame in frames]
    problem = "the octets are not whole frames"
    try:
        link.parse(octets)
    except link.LinkFrameError as error:
        problem = str(error)
    entry_id, self._next = self._next, self._next + 1
    entry = Entry(
        id=entry_id,
        at=self._clock(),
        direction=SENT,
        octets=octets,
        link=_malformed(octets, problem),
        transport=None,
        application=None,
        summary=f"not a valid frame: {problem}",
        connection=self._connection,
    )
    return [self._keep(entry)]

since ¶

since(after: int = 0) -> list[Entry]

Every entry kept whose id is greater than after, oldest first.

Source code in src/py1815/master/trace.py

def since(self, after: int = 0) -> list[Entry]:
    """Every entry kept whose id is greater than ``after``, oldest first."""
    return [entry for entry in self._entries if entry.id > after]

clear ¶

clear() -> None

Forget the frames kept. Ids carry on from where they were.

Source code in src/py1815/master/trace.py

def clear(self) -> None:
    """Forget the frames kept. Ids carry on from where they were."""
    self._entries.clear()

reset ¶

reset(
    *,
    local: Address | None = None,
    peer: Address | None = None,
) -> None

Start a new connection: forget half a frame and half a fragment.

Parameters:

Name	Type	Description	Default
`local`	`Address \| None`	This master's address and port on the new connection.	`None`
`peer`	`Address \| None`	The outstation's address and port.	`None`

Source code in src/py1815/master/trace.py

def reset(self, *, local: Address | None = None, peer: Address | None = None) -> None:
    """Start a new connection: forget half a frame and half a fragment.

    Args:
        local: This master's address and port on the new connection.
        peer: The outstation's address and port.
    """
    self._readers = {SENT: link.FrameReader(), RECEIVED: link.FrameReader()}
    for reassembler in self._reassemblers.values():
        reassembler.reset()
    self._connection += 1
    # Keep the endpoints of the connections a kept frame crossed, and of
    # the new one, and no others. A connection that closed before any
    # frame would otherwise stay here for as long as the master runs.
    needed = {entry.connection for entry in self._entries}
    for connection in [number for number in self._endpoints if number not in needed]:
        del self._endpoints[connection]
    self._endpoints[self._connection] = (local, peer)

endpoints ¶

endpoints(
    connection: int,
) -> tuple[Address | None, Address | None]

Return this master's and the outstation's address on a connection.

Either is None where it is not known: for frames recorded before any connection, and for a connection whose addresses were not given.

Source code in src/py1815/master/trace.py

def endpoints(self, connection: int) -> tuple[Address | None, Address | None]:
    """Return this master's and the outstation's address on a connection.

    Either is None where it is not known: for frames recorded before any
    connection, and for a connection whose addresses were not given.
    """
    return self._endpoints.get(connection, (None, None))

capture ¶

capture(*, after: int = 0, port: int = DNP3_PORT) -> bytes

Return the frames kept as a pcap file.

Parameters:

Name	Type	Description	Default
`after`	`int`	Only frames whose id is greater.	`0`
`port`	`int`	The outstation's port, for a connection whose addresses are not known.	`DNP3_PORT`

Source code in src/py1815/master/trace.py

def capture(self, *, after: int = 0, port: int = DNP3_PORT) -> bytes:
    """Return the frames kept as a pcap file.

    Args:
        after: Only frames whose id is greater.
        port: The outstation's port, for a connection whose addresses are
            not known.
    """
    capture = Capture()
    recorder = Recorder(capture, self, port=port)
    for entry in self.since(after):
        recorder.record(entry)
    recorder.close()
    return capture.pcap()

Recorder ¶

Write a trace's frames into a capture, one TCP connection per connection.

Give :meth:record each entry in order, or add it to the trace's listeners to write frames as they are recorded. A connection's handshake is written before its first frame, and its FIN exchange when a frame of the next connection arrives or :meth:close is called, at the time of its last frame.

A frame is one TCP segment. An address that is not IPv4, or that is not known, is written as 127.0.0.1; a port that is not known is port for the outstation and one from 41000 up for the master.

Source code in src/py1815/master/trace.py

class Recorder:
    """Write a trace's frames into a capture, one TCP connection per connection.

    Give :meth:`record` each entry in order, or add it to the trace's
    ``listeners`` to write frames as they are recorded. A connection's
    handshake is written before its first frame, and its FIN exchange when a
    frame of the next connection arrives or :meth:`close` is called, at the
    time of its last frame.

    A frame is one TCP segment. An address that is not IPv4, or that is not
    known, is written as ``127.0.0.1``; a port that is not known is ``port``
    for the outstation and one from 41000 up for the master.
    """

    def __init__(self, capture: Capture, trace: Trace, *, port: int = DNP3_PORT) -> None:
        """
        Args:
            capture: Where the packets go.
            trace: The trace the entries come from, for each connection's addresses.
            port: The outstation's port, for a connection whose addresses are
                not known.
        """
        self._capture = capture
        self._trace = trace
        self._port = port
        self._stream: Stream | None = None
        self._connection: int | None = None
        self._generation = capture.generation
        self._last = 0.0

    def record(self, entry: Entry) -> None:
        """Write one entry, opening a new connection first when it crossed one.

        Before each entry the capture may start a new file. A connection that
        was open in the old file then starts again with a handshake in the new
        one, so each file reads on its own. The old file has no FIN for it.
        """
        self._capture.boundary()
        if self._capture.generation != self._generation:
            self._generation = self._capture.generation
            self._stream = None
            self._connection = None
        if self._stream is None or entry.connection != self._connection:
            self.close()
            self._stream = self._open(entry.connection, entry.at)
            self._connection = entry.connection
        if entry.direction == SENT:
            self._stream.sent(entry.octets, entry.at)
        else:
            self._stream.received(entry.octets, entry.at)
        self._last = entry.at

    def close(self) -> None:
        """Write the FIN exchange of the open connection, if there is one.

        A connection opened in a file the capture has since rotated away from
        is forgotten instead: its FIN packets would land in the new file with
        no handshake before them.
        """
        if self._stream is not None and self._capture.generation == self._generation:
            self._stream.close(self._last)
        self._stream = None
        self._connection = None

    def abandon(self) -> None:
        """Forget the open connection without writing anything more."""
        self._stream = None
        self._connection = None

    def _open(self, connection: int, at: float) -> Stream:
        local, peer = self._trace.endpoints(connection)
        fallback = PLACEHOLDER_PORT + connection % (0x10000 - PLACEHOLDER_PORT)
        client = _address(local, fallback)
        server = _address(peer, self._port)
        stream = self._capture.stream(client=client, server=server)
        stream.open(at)
        return stream

record ¶

record(entry: Entry) -> None

Write one entry, opening a new connection first when it crossed one.

Before each entry the capture may start a new file. A connection that was open in the old file then starts again with a handshake in the new one, so each file reads on its own. The old file has no FIN for it.

Source code in src/py1815/master/trace.py

def record(self, entry: Entry) -> None:
    """Write one entry, opening a new connection first when it crossed one.

    Before each entry the capture may start a new file. A connection that
    was open in the old file then starts again with a handshake in the new
    one, so each file reads on its own. The old file has no FIN for it.
    """
    self._capture.boundary()
    if self._capture.generation != self._generation:
        self._generation = self._capture.generation
        self._stream = None
        self._connection = None
    if self._stream is None or entry.connection != self._connection:
        self.close()
        self._stream = self._open(entry.connection, entry.at)
        self._connection = entry.connection
    if entry.direction == SENT:
        self._stream.sent(entry.octets, entry.at)
    else:
        self._stream.received(entry.octets, entry.at)
    self._last = entry.at

close ¶

close() -> None

Write the FIN exchange of the open connection, if there is one.

A connection opened in a file the capture has since rotated away from is forgotten instead: its FIN packets would land in the new file with no handshake before them.

Source code in src/py1815/master/trace.py

def close(self) -> None:
    """Write the FIN exchange of the open connection, if there is one.

    A connection opened in a file the capture has since rotated away from
    is forgotten instead: its FIN packets would land in the new file with
    no handshake before them.
    """
    if self._stream is not None and self._capture.generation == self._generation:
        self._stream.close(self._last)
    self._stream = None
    self._connection = None

abandon ¶

abandon() -> None

Forget the open connection without writing anything more.

Source code in src/py1815/master/trace.py

def abandon(self) -> None:
    """Forget the open connection without writing anything more."""
    self._stream = None
    self._connection = None

iin_names ¶

iin_names(iin: IIN) -> list[str]

The indications that are set, by name, first octet first.

Source code in src/py1815/master/trace.py

def iin_names(iin: IIN) -> list[str]:
    """The indications that are set, by name, first octet first."""
    names = [bit.name for bit in IINBit if iin.first & bit]
    return names + [bit.name for bit in IIN2Bit if iin.second & bit]

The service¶

py1815.master.service ¶

The master as a service: its operations as JSON, for a caller in another process.

One table of operations, each taking a JSON object and returning one. Two ways in carry the same messages: a line of JSON at a time over a local TCP socket, for a test rig, and HTTP for a browser, where a request is a POST and what happens unasked arrives as server-sent events. Neither needs anything the standard library does not have.

The console speaks only this. Whatever it can do, a script can do with the same words.

Condition ¶

A condition wait_for is given as JSON, made a test of the store.

value holds once a point has been reported, and has the value named when one is: equals, within tolerance of it, at_least or at_most. event holds once an event arrives after the wait began, of the type and index named when they are. indication holds once the indication named is set, or clear when set is false.

Source code in src/py1815/master/service.py

class Condition:
    """A condition ``wait_for`` is given as JSON, made a test of the store.

    ``value`` holds once a point has been reported, and has the value named
    when one is: ``equals``, within ``tolerance`` of it, ``at_least`` or
    ``at_most``. ``event`` holds once an event arrives after the wait began,
    of the type and index named when they are. ``indication`` holds once the
    indication named is set, or clear when ``set`` is false.
    """

    def __init__(self, kind: str, given: Any, outstation: Outstation) -> None:
        self._outstation = outstation
        #: The point a ``value`` condition is about.
        self.point: tuple[PointType, int] | None = None
        #: The events that satisfied an ``event`` condition.
        self.matched: tuple[DecodedObject, ...] = ()
        tests = {"value": self._value, "event": self._event, "indication": self._indication}
        self._test = tests[kind](given)

    def __call__(self, store: Store) -> bool:
        return self._test(store)

    @staticmethod
    def _point(given: Mapping[str, Any], where: str, *, required: bool) -> tuple[Any, Any]:
        kind, index = given.get("type"), given.get("index")
        if kind is None and index is None and not required:
            return None, None
        if kind not in _POINT_TYPES:
            raise BadRequest(f"{where}.type is one of {', '.join(_POINT_TYPES)}")
        if index is None and not required:
            return _POINT_TYPES[kind], None
        if isinstance(index, bool) or not isinstance(index, int) or not 0 <= index <= 0xFFFF:
            raise BadRequest(f"{where}.index is 0 to 65535")
        return _POINT_TYPES[kind], index

    def _value(self, given: Any) -> Callable[[Store], bool]:
        given = _keys(
            given, "value", ("type", "index", "equals", "tolerance", "at_least", "at_most")
        )
        point, index = self._point(given, "value", required=True)
        self.point = (point, index)
        equals = given.get("equals")
        if equals is not None and not isinstance(equals, bool):
            equals = _number_given(equals, "value.equals")
        tolerance = _number_given(given.get("tolerance", 0), "value.tolerance")
        if tolerance < 0:
            raise BadRequest("value.tolerance is zero or more")
        bounds = [
            (name, _number_given(given[name], f"value.{name}"))
            for name in ("at_least", "at_most")
            if given.get(name) is not None
        ]

        def test(store: Store) -> bool:
            stored = store.get(point, index)
            if stored is None:
                return False
            value = stored.value
            numeric = isinstance(value, (int, float)) and not isinstance(value, bool)
            if equals is not None:
                if isinstance(equals, bool) or not numeric:
                    if value != equals:
                        return False
                elif not abs(float(value) - equals) <= tolerance:  # type: ignore[arg-type]
                    return False
            for name, bound in bounds:
                if not numeric:
                    return False
                if name == "at_least" and not value >= bound:  # type: ignore[operator]
                    return False
                if name == "at_most" and not value <= bound:  # type: ignore[operator]
                    return False
            return True

        return test

    def _event(self, given: Any) -> Callable[[Store], bool]:
        given = _keys(given, "event", ("type", "index"))
        point, index = self._point(given, "event", required=False)
        started = self._outstation.store.events_received

        def test(store: Store) -> bool:
            self.matched = tuple(
                event
                for event in store.events_since(started)
                if (point is None or event.point is point)
                and (index is None or event.index == index)
            )
            return bool(self.matched)

        return test

    def _indication(self, given: Any) -> Callable[[Store], bool]:
        given = _keys(given, "indication", ("name", "set"))
        name = given.get("name")
        if name not in INDICATIONS:
            raise BadRequest(f"indication.name is one of {', '.join(INDICATIONS)}")
        bit = INDICATIONS[name]
        wanted = given.get("set", True)
        if not isinstance(wanted, bool):
            raise BadRequest("indication.set is true or false")
        outstation = self._outstation

        def test(_store: Store) -> bool:
            # With no response yet, the bit is neither set nor clear.
            indications = outstation.indications
            return indications is not None and indications.is_set(bit) == wanted

        return test

ControlNotAllowed ¶

Bases: Exception

The operation commands an outstation, and this service was not started to.

Source code in src/py1815/master/service.py

class ControlNotAllowed(Exception):
    """The operation commands an outstation, and this service was not started to."""

BadRequest ¶

Bases: ValueError

A message that does not name an operation, or gives it the wrong things.

Source code in src/py1815/master/service.py

class BadRequest(ValueError):
    """A message that does not name an operation, or gives it the wrong things."""

Service ¶

A master, and the operations it answers as JSON.

Source code in src/py1815/master/service.py

class Service:
    """A master, and the operations it answers as JSON."""

    def __init__(
        self,
        master: Master | None = None,
        *,
        allow_control: bool = False,
        capture: CaptureFile | None = None,
    ) -> None:
        """
        Args:
            master: The master to serve. A new one when not given.
            allow_control: Carry out the operations that command an
                outstation: its outputs, its counters, its clock, its restart
                indication, and any request by function code other than a
                read. Off unless asked for. A master that is pointed at real
                equipment should have to be told before it can change it.
                When off, outstations are also added with the automatic
                tasks that write (restart clear, time write) disabled.
            capture: A pcap file to write every frame of every outstation
                added to, as it is sent or received. The service closes it
                when it closes.
        """
        self.master = Master() if master is None else master
        self.allow_control = allow_control
        self.capture_file = capture
        self._recorders: dict[str, Recorder] = {}
        #: Each outstation's listener that writes the capture file, by name.
        self._capture_listeners: dict[str, Callable[[Entry], None]] = {}
        self._names: dict[str, dict[PointType, dict[int, str]]] = {}
        self._profiles: dict[str, PointMap] = {}
        self._tls: dict[str, TlsSettings] = {}
        self._der_profiles: dict[str, DerProfile] = {}
        self._device_profiles: dict[str, str] = {}
        #: The outstations an evaluation is running against.
        self._evaluating: set[str] = set()
        self._subscribers: set[asyncio.Queue[dict[str, Any]]] = set()
        self.stopped = asyncio.Event()
        self._operations: dict[str, Callable[[Mapping[str, Any]], Awaitable[Any]]] = {
            "status": self._status,
            "profile": self._profile,
            "add": self._add,
            "remove": self._remove,
            "connect": self._connect,
            "disconnect": self._disconnect,
            "idle": self._idle,
            "set_tasks": self._set_tasks,
            "wait_for": self._wait_for,
            "scan": self._scan,
            "read": self._read,
            "values": self._values,
            "events": self._events,
            "request": self._request,
            "operate": self._operate,
            "write_time": self._write_time,
            "synchronize_time": self._synchronize_time,
            "clear_restart": self._clear_restart,
            "freeze": self._freeze,
            "restart": self._restart,
            "broadcast": self._broadcast,
            "enable_unsolicited": self._enable_unsolicited,
            "disable_unsolicited": self._disable_unsolicited,
            "repeat": self._repeat,
            "trace": self._trace,
            "capture": self._capture,
            "clear": self._clear,
            "deviate": self._deviate,
            "checks": self._checks,
            "evaluate": self._evaluate,
            "der.read": self._der_read,
            "der.write": self._der_write,
            "der.enable": self._der_enable,
            "der.disable": self._der_disable,
            "der.functions": self._der_functions,
            "der.curve": self._der_curve,
            "der.write_curve": self._der_write_curve,
            "der.compare": self._der_compare,
            "der.document": self._der_document,
            "stop": self._stop,
        }

    # -------------------------------------------------------------- messages

    @property
    def operations(self) -> tuple[str, ...]:
        """The names of the operations, as a message's ``op`` may give them."""
        return tuple(self._operations)

    async def handle(self, message: Any) -> dict[str, Any]:
        """Answer one message. Never raises: what went wrong is the answer."""
        identifier = message.get("id") if isinstance(message, Mapping) else None
        try:
            if not isinstance(message, Mapping):
                raise BadRequest("a message is a JSON object")
            operation = self._operations.get(str(message.get("op")))
            if operation is None:
                raise BadRequest(
                    f"{message.get('op')!r} is not an operation; one of "
                    + ", ".join(self._operations)
                )
            params = message.get("params") or {}
            if not isinstance(params, Mapping):
                raise BadRequest("params is a JSON object")
            if "outstation" in message and "outstation" not in params:
                params = {**params, "outstation": message["outstation"]}
            _commanded.set(False)
            result = await operation(params)
            if _commanded.get():
                self._audit(str(message.get("op")), params, _outcome(result))
        except ControlNotAllowed as error:
            logger.warning(
                "dnp3 master: refused %s for %s: commanding is off",
                message.get("op") if isinstance(message, Mapping) else None,
                params.get("outstation") if isinstance(params, Mapping) else None,
            )
            return {
                "id": identifier,
                "ok": False,
                "error": {"kind": "not_allowed", "message": str(error)},
            }
        # Busy, which the association raises for a second request while one is
        # outstanding, is deliberately not caught. An outstation takes its
        # requests one at a time through a lock, so no message can cause it; if
        # it is ever raised here the fault is in this library and should be seen.
        except (ValueError, KeyError, TypeError) as error:
            text = str(error.args[0]) if isinstance(error, KeyError) and error.args else str(error)
            return {"id": identifier, "ok": False, "error": {"kind": "request", "message": text}}
        except OSError as error:
            if _commanded.get():
                self._audit(str(message.get("op")), params, f"failed: {error}")
            return {
                "id": identifier,
                "ok": False,
                "error": {"kind": "connection", "message": str(error) or type(error).__name__},
            }
        return {"id": identifier, "ok": True, "result": result}

    @staticmethod
    def _audit(op: str, params: Mapping[str, Any], outcome: str) -> None:
        """Log a command sent to an outstation, with what was asked and what came of it."""
        asked = {key: value for key, value in params.items() if key != "outstation"}
        text = json.dumps(asked, separators=(",", ":"), default=str)
        logger.info(
            "dnp3 master: command %s to %s %s: %s", op, params.get("outstation"), text, outcome
        )

    # ---------------------------------------------------------- subscription

    def subscribe(self) -> asyncio.Queue[dict[str, Any]]:
        """A queue that receives everything that happens, until unsubscribed."""
        queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue(maxsize=SUBSCRIBER_QUEUE)
        self._subscribers.add(queue)
        return queue

    def unsubscribe(self, queue: asyncio.Queue[dict[str, Any]]) -> None:
        self._subscribers.discard(queue)

    def publish(self, event: dict[str, Any]) -> None:
        """Send an event to every subscriber.

        A subscriber that has fallen SUBSCRIBER_QUEUE updates behind has
        its waiting updates replaced by one lost event that says how many
        were dropped, and stays subscribed. It can then reload what it shows,
        and memory stays bounded however long it lags.
        """
        for queue in list(self._subscribers):
            try:
                queue.put_nowait(event)
            except asyncio.QueueFull:
                dropped = 1
                while not queue.empty():
                    waiting = queue.get_nowait()
                    # A lost update already waiting stands for all it dropped.
                    dropped += waiting.get("dropped", 1) if waiting.get("event") == "lost" else 1
                logger.warning(
                    "dnp3 master: a subscriber fell behind; dropped %d update(s)", dropped
                )
                queue.put_nowait({"event": "lost", "dropped": dropped})

    # ----------------------------------------------------------- outstations

    def set_names(self, outstation: str, names: Names) -> None:
        """Give an outstation's points names, for a caller that knows its map."""
        self._names[outstation] = {point: dict(indexed) for point, indexed in names.items()}

    def set_profile(self, outstation: str, point_map: PointMap) -> None:
        """Give an outstation the profile it is meant to serve.

        The whole map, and not only the points the outstation reports. That
        is what lets a caller see the difference: a point of the profile the
        outstation has never reported is one it does not implement, or one
        nobody has read yet, and a master cannot tell which from silence.
        """
        self._profiles[outstation] = point_map
        self._der_profiles.pop(outstation, None)
        names: dict[PointType, dict[int, str]] = {point: {} for point in PointType}
        for (kind, index), point in point_map.points.items():
            for point_type in _KIND_TYPES[kind]:
                names[point_type][index] = point.name
        self._names[outstation] = names

    def set_device_profile(self, outstation: str, document: str) -> None:
        """Give an outstation its Device Profile document.

        ``der.compare`` compares what the outstation serves with it, and
        ``der.document`` lays it out for reading.

        Raises ValueError for a document whose point lists cannot be read.
        """
        read_device_profile(document)
        self._device_profiles[outstation] = document

    def _der_profile(self, outstation: str) -> DerProfile:
        found = self._der_profiles.get(outstation)
        if found is None:
            point_map = self._profiles.get(outstation)
            if point_map is None:
                raise BadRequest(
                    f"{outstation} was given no profile; add it with --profile to name its points"
                )
            found = self._der_profiles[outstation] = DerProfile(point_map)
        return found

    def _name(self, outstation: str, point: PointType | None, index: int | None) -> str | None:
        if point is None or index is None:
            return None
        return self._names.get(outstation, {}).get(point, {}).get(index)

    def _profile_counts(self, outstation: str) -> dict[str, int] | None:
        point_map = self._profiles.get(outstation)
        if point_map is None:
            return None
        counts = {point.value: 0 for point in PointType}
        for kind, _index in point_map.points:
            for point_type in _KIND_TYPES[kind]:
                counts[point_type.value] += 1
        return counts

    def _outstation(self, params: Mapping[str, Any]) -> Outstation:
        name = params.get("outstation")
        if name is None:
            raise BadRequest("the operation needs an outstation")
        try:
            return self.master[str(name)]
        except KeyError:
            raise BadRequest(f"there is no outstation named {name!r}") from None

    def _describe_outstation(self, outstation: Outstation) -> dict[str, Any]:
        last = outstation.last_response
        return {
            "name": outstation.name,
            "host": outstation.host,
            "port": outstation.port,
            "outstation_address": outstation.association.outstation_address,
            "master_address": outstation.association.master_address,
            "read_retries": outstation.association.read_retries,
            "tls": self._tls[outstation.name].describe() if outstation.name in self._tls else None,
            "connected": outstation.connected,
            "indications": iin_names(last.iin) if last is not None and last.iin else [],
            "last_response_at": outstation.last_response_at,
            "counts": dict(outstation.counts),
            "unsolicited": len(outstation.unsolicited),
            "points": {point.value: len(outstation.store.points(point)) for point in PointType},
            "events": len(outstation.store.events),
            "frames": len(outstation.trace),
            "repeat": outstation.scan_intervals,
            "tasks": outstation.tasks.describe(),
            "tasks_due": list(outstation.tasks_due),
            "reconnect": outstation.reconnect,
            "named": bool(self._names.get(outstation.name)),
            "profile": self._profile_counts(outstation.name),
            "device_profile": outstation.name in self._device_profiles,
        }

    def _describe_exchange(self, outstation: Outstation, exchange: Exchange) -> dict[str, Any]:
        objects = [
            describe_object(decoded, self._name(outstation.name, decoded.point, decoded.index))
            for decoded in exchange.objects[:_OBJECTS_DESCRIBED]
        ]
        return {
            "function": exchange.function.name,
            "task": exchange.task,
            "sequence": exchange.sequence,
            "outcome": exchange.outcome.value,
            "fragments": len(exchange.fragments),
            "indications": iin_names(exchange.iin) if exchange.iin is not None else None,
            "objects": objects,
            "object_count": len(exchange.objects),
            "undecoded": [
                {"problem": decoded.problem, "unread": decoded.unread.hex()}
                for decoded in exchange.undecoded
            ],
            "elapsed_ms": round(exchange.elapsed * 1000, 1),
            "request": exchange.request.hex(),
            "retries": exchange.retries,
            "broadcast": None if exchange.broadcast is None else exchange.broadcast.name.lower(),
        }

    def attach(self, outstation: Outstation) -> None:
        """Have what happens to an outstation published to subscribers."""
        name = outstation.name

        def on_frame(entry: Entry) -> None:
            self.publish({"event": "frame", "outstation": name, "frame": describe_entry(entry)})

        def on_exchange(exchange: Exchange) -> None:
            described = self._describe_exchange(outstation, exchange)
            self.publish({"event": "exchange", "outstation": name, "exchange": described})

        def on_unsolicited(unsolicited: Unsolicited) -> None:
            objects = [
                describe_object(decoded, self._name(name, decoded.point, decoded.index))
                for decoded in unsolicited.objects
            ]
            self.publish(
                {
                    "event": "unsolicited",
                    "outstation": name,
                    "indications": iin_names(unsolicited.iin),
                    "objects": objects,
                }
            )

        def on_connection(connected: bool) -> None:
            self.publish({"event": "connection", "outstation": name, "connected": connected})

        outstation.trace.listeners.append(on_frame)
        if self.capture_file is not None:
            recorder = Recorder(self.capture_file, outstation.trace, port=outstation.port)
            self._recorders[name] = recorder

            def on_frame_captured(entry: Entry) -> None:
                # A listener's error reaches the code that recorded the frame,
                # which is the receive loop or a request. A full disk must not
                # end the DNP3 connection, so the capture stops instead.
                try:
                    recorder.record(entry)
                except (OSError, ValueError):
                    logger.exception("dnp3 master: writing the capture of %s failed", name)
                    self._stop_capture()

            self._capture_listeners[name] = on_frame_captured
            outstation.trace.listeners.append(on_frame_captured)
        outstation.on_exchange = on_exchange
        outstation.on_unsolicited = on_unsolicited
        outstation.on_connection = on_connection

    # ------------------------------------------------------------ operations

    async def _status(self, _params: Mapping[str, Any]) -> dict[str, Any]:
        return {
            "allow_control": self.allow_control,
            "outstations": [
                self._describe_outstation(outstation)
                for outstation in self.master.outstations.values()
            ],
        }

    async def _profile(self, params: Mapping[str, Any]) -> dict[str, Any]:
        """Every point of the profile an outstation was given, reported or not."""
        outstation = self._outstation(params)
        point_map = self._profiles.get(outstation.name)
        if point_map is None:
            return {"version": None, "points": None}
        points: dict[str, list[dict[str, Any]]] = {point.value: [] for point in PointType}
        for (kind, index), point in sorted(
            point_map.points.items(), key=lambda item: (item[0][0].value, item[0][1])
        ):
            label, enumeration = split_enumeration(point.name)
            for point_type in _KIND_TYPES[kind]:
                points[point_type.value].append(
                    {
                        "index": index,
                        "name": point.name,
                        "label": label,
                        "enumeration": enumeration,
                        "mandatory": point.mandatory,
                        "section": point.section,
                    }
                )
        return {"version": point_map.profile_version, "points": points}

    async def _add(self, params: Mapping[str, Any]) -> dict[str, Any]:
        name = params.get("name")
        if not name or not isinstance(name, str):
            raise BadRequest("an outstation is added with a name")
        if "host" not in params:
            raise BadRequest("an outstation is added with a host")
        options: dict[str, Any] = {"host": str(params["host"])}
        for key, kind in (
            ("port", int),
            ("outstation_address", int),
            ("master_address", int),
            ("response_timeout", float),
            ("connect_timeout", float),
        ):
            if params.get(key) is not None:
                options[key] = kind(params[key])
        if params.get("read_retries") is not None:
            retries = params["read_retries"]
            if isinstance(retries, bool) or not isinstance(retries, int) or retries < 0:
                raise BadRequest("read_retries is a count of retries, zero or more")
            options["read_retries"] = retries
        if params.get("confirm") is not None:
            options["confirm"] = bool(params["confirm"])
        if "reconnect" in params:
            given = params["reconnect"]
            options["reconnect"] = None if given is None else float(given)
        tls = self._tls_settings(params.get("tls"))
        if tls is not None:
            options["tls"] = self._tls_context(tls)
            options["server_name"] = tls.server_name
        options["manual"] = manual = bool(params.get("manual", False))
        options["tasks"] = self._tasks(params.get("tasks"), manual)
        outstation = await self.master.add(name, connect=False, **options)
        if tls is not None:
            self._tls[name] = tls
        self.attach(outstation)
        intervals = {
            "integrity": params.get("integrity_interval"),
            "events": params.get("event_interval"),
            # Output status is not in an integrity poll, so one that is repeated
            # would leave the outputs unread. They are read as often, unless
            # the caller says how often, or says not to with null.
            "outputs": params.get("output_interval", params.get("integrity_interval")),
        }
        for kind_name, interval in intervals.items():
            if interval is not None:
                outstation.repeat_scan(kind_name, float(interval))
        self.publish({"event": "outstations"})
        if params.get("connect", True):
            try:
                await outstation.connect()
            except OSError:
                # Added, and not connected: the caller can see it and try again.
                self.publish({"event": "connection", "outstation": name, "connected": False})
                raise
        return self._describe_outstation(outstation)

    @staticmethod
    def _tls_settings(given: Any) -> TlsSettings | None:
        if given is None:
            return None
        if not isinstance(given, Mapping):
            raise BadRequest("tls is an object of files and a name, or null for plain TCP")
        return TlsSettings.from_mapping(given)

    @staticmethod
    def _tls_context(tls: TlsSettings) -> ssl.SSLContext:
        """Make the context, and say a file that cannot be used is the caller's mistake."""
        try:
            return tls.context()
        except (OSError, ssl.SSLError) as error:
            # A missing or unreadable file, and not a connection that failed.
            raise BadRequest(f"tls: {error}") from None

    def _tasks(self, given: Any, manual: bool) -> Tasks:
        """Build the task settings for a new outstation from the ``add`` parameters."""
        if given is not None and not isinstance(given, Mapping):
            raise BadRequest("tasks is an object of choices by task")
        tasks = (Tasks.none() if manual else Tasks()).changed(given or {})
        if self.allow_control:
            return tasks
        # A read-only service does not run the tasks that write. Requesting
        # one explicitly is refused; otherwise they are disabled, and the
        # outstation's status shows that.
        for name in WRITING:
            if given and given.get(name):
                self._commanding(f"the {name} task")
        return tasks.reading_only()

    def _stop_capture(self) -> None:
        """Stop writing the capture file for every outstation, after a write failed.

        The file is shared, so a failure for one outstation is a failure for
        all. Open connections are abandoned without their FIN packets, since
        writing more to a failing file would only fail again.
        """
        for name, listener in self._capture_listeners.items():
            with contextlib.suppress(KeyError, ValueError):
                self.master[name].trace.listeners.remove(listener)
        self._capture_listeners.clear()
        for recorder in self._recorders.values():
            recorder.abandon()
        self._recorders.clear()
        if self.capture_file is not None:
            with contextlib.suppress(OSError):
                self.capture_file.close()

    async def _remove(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        await self.master.remove(outstation.name)
        self._capture_listeners.pop(outstation.name, None)
        recorder = self._recorders.pop(outstation.name, None)
        if recorder is not None:
            recorder.close()
        self._names.pop(outstation.name, None)
        self._profiles.pop(outstation.name, None)
        self._tls.pop(outstation.name, None)
        self._der_profiles.pop(outstation.name, None)
        self._device_profiles.pop(outstation.name, None)
        self.publish({"event": "outstations"})
        return {"removed": outstation.name}

    async def _connect(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        wait = params.get("wait")
        if wait is not None and (
            isinstance(wait, bool)
            or not isinstance(wait, (int, float))
            or not math.isfinite(wait)
            or wait < 0
        ):
            raise BadRequest("wait is seconds to keep trying, zero or more")
        await outstation.connect(wait=wait)
        return self._describe_outstation(outstation)

    async def _disconnect(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        await outstation.close()
        self.publish({"event": "connection", "outstation": outstation.name, "connected": False})
        return self._describe_outstation(outstation)

    async def _idle(self, params: Mapping[str, Any]) -> dict[str, Any]:
        """Wait until no automatic task is pending or running."""
        outstation = self._outstation(params)
        await outstation.idle()
        return self._describe_outstation(outstation)

    async def _set_tasks(self, params: Mapping[str, Any]) -> dict[str, Any]:
        """Change what the master does for an outstation unasked."""
        outstation = self._outstation(params)
        given = params.get("tasks")
        if not isinstance(given, Mapping) or not given:
            raise BadRequest("tasks is an object of choices by task")
        # Refused for what is wrong with it before it is refused for not
        # being allowed, so the two are told apart.
        tasks = outstation.tasks.changed(given)
        if not self.allow_control:
            for name in WRITING:
                if given.get(name):
                    self._commanding(f"the {name} task")
        outstation.tasks = tasks
        self.publish({"event": "outstations"})
        return self._describe_outstation(outstation)

    async def _wait_for(self, params: Mapping[str, Any]) -> dict[str, Any]:
        """Wait until a condition on the store, an event or an indication holds."""
        outstation = self._outstation(params)
        timeout = params.get("timeout")
        if (
            isinstance(timeout, bool)
            or not isinstance(timeout, (int, float))
            or not math.isfinite(timeout)
            or timeout < 0
        ):
            raise BadRequest("timeout is seconds to wait, zero or more")
        named = [kind for kind in ("value", "event", "indication") if kind in params]
        if len(named) != 1:
            raise BadRequest("a wait names one condition: value, event or indication")
        condition = Condition(named[0], params[named[0]], outstation)
        started = time.monotonic()
        held = await outstation.wait_for(condition, float(timeout))
        result: dict[str, Any] = {
            "held": held,
            "elapsed_ms": round((time.monotonic() - started) * 1000, 1),
            "indications": iin_names(outstation.indications) if outstation.indications else [],
            "point": None,
            "events": [
                describe_object(event, self._name(outstation.name, event.point, event.index))
                for event in condition.matched
            ],
        }
        if condition.point is not None:
            point, index = condition.point
            value = outstation.store.get(point, index)
            if value is not None:
                name = self._name(outstation.name, point, index)
                result["point"] = describe_value(index, value, point, name)
        return result

    async def _scan(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        exchange = await outstation.scan(str(params.get("kind", "integrity")))
        return self._describe_exchange(outstation, exchange)

    async def _read(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        points = params.get("points")
        if not isinstance(points, Mapping) or not points:
            raise BadRequest("a read names points, by type")
        arguments = {
            "bi": "binary_inputs",
            "bo": "binary_outputs",
            "counter": "counters",
            "frozen": "frozen_counters",
            "ai": "analog_inputs",
            "ao": "analog_outputs",
        }
        named: dict[str, Any] = {}
        for kind, indices in points.items():
            if kind not in arguments:
                raise BadRequest(f"{kind!r} is not a point type; one of {', '.join(arguments)}")
            named[arguments[kind]] = (
                indices if isinstance(indices, str) else [int(index) for index in indices]
            )
        exchange = await outstation.read(**named)
        return self._describe_exchange(outstation, exchange)

    async def _values(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        wanted = params.get("types") or list(_POINT_TYPES)
        result: dict[str, list[dict[str, Any]]] = {}
        for kind in wanted:
            if kind not in _POINT_TYPES:
                raise BadRequest(f"{kind!r} is not a point type")
            point = _POINT_TYPES[kind]
            result[kind] = [
                describe_value(index, value, point, self._name(outstation.name, point, index))
                for index, value in outstation.store.points(point).items()
            ]
        return {"points": result}

    async def _events(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        limit = int(params.get("limit", 500))
        events = outstation.store.events[-limit:] if limit > 0 else ()
        return {
            "events": [
                describe_object(event, self._name(outstation.name, event.point, event.index))
                for event in events
            ],
            "total": len(outstation.store.events),
        }

    @staticmethod
    def _function_and_body(params: Mapping[str, Any]) -> tuple[FunctionCode, bytes]:
        function = params.get("function")
        try:
            code = (
                FunctionCode[function.upper()]
                if isinstance(function, str)
                else FunctionCode(int(function))  # type: ignore[arg-type]
            )
        except (KeyError, ValueError, TypeError):
            raise BadRequest(f"{function!r} is not a function code") from None
        try:
            body = bytes.fromhex(str(params.get("body", "")))
        except ValueError:
            raise BadRequest("body is the octets after the function code, in hexadecimal") from None
        return code, body

    async def _request(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        code, body = self._function_and_body(params)
        if code not in requests.READING_FUNCTIONS:
            # Any other function code changes the outstation, and this is the
            # one operation that would otherwise send it unasked-about.
            self._commanding(f"a {code.name} request")
        exchange = await outstation.request(code, body)
        return self._describe_exchange(outstation, exchange)

    # ------------------------------------------------------------- commands

    def _commanding(self, what: str) -> None:
        _commanded.set(True)
        if not self.allow_control:
            raise ControlNotAllowed(
                f"{what} commands the outstation, and commanding is off. "
                "Start the service with --allow-control to turn it on."
            )

    def _describe_operated(self, outstation: Outstation, operated: Operated) -> dict[str, Any]:
        statuses = []
        for status in operated.statuses:
            command = status.command
            control = command.control
            if isinstance(control, AnalogOutput):
                value: Any = _number(control.value)
            else:
                value = control.control_code
            statuses.append(
                {
                    "type": command.point.value,
                    "index": command.index,
                    "name": self._name(outstation.name, command.point, command.index),
                    "variation": command.variation,
                    "value": value,
                    "status": None if status.status is None else status.status.name,
                    "echoed": status.echoed,
                }
            )
        return {
            "mode": operated.mode.value,
            "operated": operated.operated,
            "accepted": operated.accepted,
            "statuses": statuses,
            "exchanges": [
                self._describe_exchange(outstation, exchange) for exchange in operated.exchanges
            ],
        }

    async def _operate(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        points = params.get("points")
        if not isinstance(points, Mapping) or not points:
            raise BadRequest("an operate names outputs, by type: bo and ao")
        unknown = set(points) - {"bo", "ao"}
        if unknown:
            raise BadRequest(f"{sorted(unknown)[0]!r} is not an output; bo or ao")
        for kind, named in points.items():
            if not isinstance(named, Mapping):
                raise BadRequest(f"{kind} is an object of values by index")
        variation = params.get("variation")
        # Refused for what is wrong with it before it is refused for not
        # being allowed, so the two are told apart.
        plan = Plan(
            commands(
                points.get("bo"),
                points.get("ao"),
                variation=None if variation is None else int(variation),
            ),
            str(params.get("mode", Mode.DIRECT.value)),
        )
        self._commanding("an operate")
        # Through the outstation's own method, so the plan is carried out in
        # one turn exactly as a caller in Python has it carried out.
        operated = await outstation._carry_out(plan)  # pylint: disable=protected-access
        return self._describe_operated(outstation, operated)

    async def _write_time(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        when = params.get("time_ms")
        if when is not None and (isinstance(when, bool) or not isinstance(when, int)):
            raise BadRequest("time_ms is milliseconds since the epoch, a whole number")
        self._commanding("a time write")
        return self._describe_exchange(outstation, await outstation.write_time(when))

    async def _synchronize_time(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        procedure = params.get("procedure", LAN)
        if procedure not in PROCEDURES:
            raise BadRequest(f"{procedure!r} is not a time procedure; lan or non_lan")
        self._commanding("setting the clock")
        synchronized = await outstation.synchronize_time(str(procedure))
        return self._describe_synchronized(outstation, synchronized)

    def _describe_synchronized(
        self, outstation: Outstation, synchronized: Synchronized
    ) -> dict[str, Any]:
        delay = synchronized.delay_ms
        return {
            "procedure": synchronized.procedure,
            "written": synchronized.written,
            "accepted": synchronized.accepted,
            "delay_ms": None if delay is None else round(delay, 1),
            "time_ms": synchronized.time_ms,
            "exchanges": [
                self._describe_exchange(outstation, exchange) for exchange in synchronized.exchanges
            ],
        }

    async def _clear_restart(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        self._commanding("clearing the restart indication")
        return self._describe_exchange(outstation, await outstation.clear_restart())

    async def _freeze(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        self._commanding("a freeze")
        exchange = await outstation.freeze(
            clear=bool(params.get("clear", False)), respond=bool(params.get("respond", True))
        )
        return self._describe_exchange(outstation, exchange)

    async def _restart(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        kind = str(params.get("kind", "cold"))
        if kind not in ("cold", "warm"):
            raise BadRequest(f"{kind!r} is not a restart; cold or warm")
        self._commanding("a restart")
        return self._describe_exchange(outstation, await outstation.restart(kind))

    async def _broadcast(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        code, body = self._function_and_body(params)
        address = broadcast_address(params.get("address", DEFAULT_BROADCAST.name.lower()))
        if code not in requests.READING_FUNCTIONS:
            self._commanding(f"a {code.name} broadcast")
        exchange = await outstation.broadcast(code, body, address=address)
        return self._describe_exchange(outstation, exchange)

    @staticmethod
    def _classes(params: Mapping[str, Any]) -> list[int]:
        return [int(number) for number in params.get("classes") or (1, 2, 3)]

    async def _enable_unsolicited(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        exchange = await outstation.enable_unsolicited(*self._classes(params))
        return self._describe_exchange(outstation, exchange)

    async def _disable_unsolicited(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        exchange = await outstation.disable_unsolicited(*self._classes(params))
        return self._describe_exchange(outstation, exchange)

    async def _repeat(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        interval = params.get("interval")
        outstation.repeat_scan(
            str(params.get("kind", "integrity")), None if interval is None else float(interval)
        )
        return {"repeat": outstation.scan_intervals}

    async def _trace(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        entries = outstation.trace.since(int(params.get("after", 0)))
        limit = int(params.get("limit", 500))
        return {"frames": [describe_entry(entry) for entry in entries[-limit:]]}

    async def _capture(self, params: Mapping[str, Any]) -> dict[str, Any]:
        """Return the frames kept as a pcap file, in base64 so JSON can carry it."""
        outstation = self._outstation(params)
        after = int(params.get("after", 0))
        frames = len(outstation.trace.since(after))
        pcap = outstation.trace.capture(after=after, port=outstation.port)
        return {"frames": frames, "pcap": base64.b64encode(pcap).decode("ascii")}

    async def _clear(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        what = params.get("what", "trace")
        if what == "trace":
            outstation.trace.clear()
        elif what == "events":
            outstation.store.clear_events()
        else:
            raise BadRequest("what is cleared is the trace or the events")
        return {"cleared": what}

    async def _deviate(self, params: Mapping[str, Any]) -> dict[str, Any]:
        """Turn named deviations on or off, and report which are on.

        ``set`` is an object of deviation names to true or false; ``clear``
        turns every deviation off first. With neither, the current state is
        returned unchanged. A request with anything wrong in it changes
        nothing.
        """
        outstation = self._outstation(params)
        clear = params.get("clear", False)
        if not isinstance(clear, bool):
            raise BadRequest("clear is true or false")
        changes = params.get("set", {})
        if not isinstance(changes, Mapping):
            raise BadRequest("set is an object of deviation names to true or false")
        names = outstation.deviations.names()
        for name, state in changes.items():
            if name not in names:
                raise BadRequest(f"set.{name} is not a deviation; one of {', '.join(names)}")
            if not isinstance(state, bool):
                raise BadRequest(f"set.{name} is true or false")
        # Every name and state has been checked, so the change is made whole.
        if clear:
            outstation.deviations.clear()
        for name, state in changes.items():
            outstation.deviations.set(name, state)
        return {
            "deviations": outstation.deviations.describe(),
            "on": list(outstation.deviations.on()),
        }

    # ------------------------------------------------------------ evaluation

    async def _checks(self, _params: Mapping[str, Any]) -> dict[str, Any]:
        """List the checks ``evaluate`` can run, in the order it runs them."""
        return {"checks": [describe_check(check) for check in CATALOG]}

    async def _evaluate(self, params: Mapping[str, Any]) -> dict[str, Any]:
        """Run checks against an outstation, and return the report.

        Each result is also published as a ``check`` event as its check ends.
        A check that writes is run only when the service was started to
        command, and a check that has the master break a protocol rule only
        when ``allow_deviations`` is true. The others are reported as not run.
        """
        outstation = self._outstation(params)
        profile = self._der_profile(outstation.name)
        wanted = params.get("checks")
        if wanted is not None and (
            not isinstance(wanted, list) or not all(isinstance(each, str) for each in wanted)
        ):
            raise BadRequest("checks is a list of check names, or null for all of them")
        chosen = select(wanted)
        deviating = params.get("allow_deviations", False)
        if not isinstance(deviating, bool):
            raise BadRequest("allow_deviations is true or false")
        settle = params.get("settle", DEFAULT_SETTLE)
        if (
            isinstance(settle, bool)
            or not isinstance(settle, (int, float))
            or not math.isfinite(settle)
            or not settle > 0
        ):
            raise BadRequest("settle is a number of seconds greater than 0")
        curves = params.get("curves")
        if curves is not None and (
            isinstance(curves, bool) or not isinstance(curves, int) or curves < 1
        ):
            raise BadRequest("curves is a whole number of at least 1, or null to find out")
        if outstation.name in self._evaluating:
            raise BadRequest(f"an evaluation of {outstation.name} is already running")
        if self.allow_control and any(check.commands for check in chosen):
            # Recorded in the command log, as every operation that writes is.
            self._commanding("an evaluation that writes")

        def ended(result: Result) -> None:
            self.publish(
                {"event": "check", "outstation": outstation.name, "result": result.describe()}
            )

        self._evaluating.add(outstation.name)
        try:
            report = await evaluate(
                outstation,
                profile,
                checks=chosen,
                commanding=self.allow_control,
                deviating=deviating,
                settle=float(settle),
                curves=curves,
                device_profile=self._device_profiles.get(outstation.name),
                on_result=ended,
            )
        except NoAnswer as error:
            raise ConnectionError(f"{outstation.name}: {error}") from error
        finally:
            self._evaluating.discard(outstation.name)
        return report.describe()

    # ---------------------------------------------------------- DER profile

    def _der(self, params: Mapping[str, Any]) -> tuple[Outstation, DerProfile, Der]:
        outstation = self._outstation(params)
        profile = self._der_profile(outstation.name)
        return outstation, profile, Der(outstation, profile)

    @staticmethod
    def _mode(params: Mapping[str, Any]) -> str:
        return str(params.get("mode", Mode.DIRECT.value))

    def _describe_curve(self, outstation: Outstation, curve: Curve) -> dict[str, Any]:
        fields = {position: reading.point for position, reading in enumerate(curve.fields)}
        return {
            "number": curve.number,
            "type": curve.type,
            "type_name": _enumerated(fields[1], curve.type) if 1 in fields else None,
            "count": curve.count,
            "x_units": curve.x_units,
            "x_units_name": _enumerated(fields[3], curve.x_units) if 3 in fields else None,
            "y_units": curve.y_units,
            "y_units_name": _enumerated(fields[4], curve.y_units) if 4 in fields else None,
            "points": [[_number(x), _number(y)] for x, y in curve.points],
            "referenced": curve.referenced,
            "exchanges": [
                self._describe_exchange(outstation, exchange)
                for read in curve.reads
                for exchange in read.exchanges
            ],
        }

    async def _der_read(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation, profile, der = self._der(params)
        names = params.get("names") or []
        if isinstance(names, str) or not isinstance(names, list):
            raise BadRequest("names is a list of point names or addresses")
        group = params.get("group")
        if not names and not group:
            raise BadRequest("a read names points, or a group")
        points = [profile.point(str(name)) for name in names]
        if group:
            points += [point for point in profile.group(str(group)) if point not in points]
        readings = await der.run(read_plan(points))
        return {
            "points": [describe_reading(reading) for reading in readings.readings],
            "exchanges": [
                self._describe_exchange(outstation, exchange) for exchange in readings.exchanges
            ],
        }

    async def _der_write(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation, profile, der = self._der(params)
        points = params.get("points")
        if not isinstance(points, Mapping) or not points:
            raise BadRequest("a write names outputs, as an object of values by name")
        variation = params.get("variation")
        plan = write_plan(
            profile,
            points,
            verify=bool(params.get("verify", False)),
            mode=self._mode(params),
            variation=None if variation is None else int(variation),
        )
        self._commanding("a profile write")
        written = await der.run(plan)
        described = []
        for setpoint in written.setpoints:
            point = setpoint.point
            described.append(
                {
                    "address": point_address(point),
                    "type": _KIND_TYPES[point.kind][0].value,
                    "index": point.index,
                    "name": point.name,
                    "units": _units(point),
                    "requested": _number(setpoint.requested),
                    "sent": _number(setpoint.sent),
                    "sent_value": _number(setpoint.sent_value),
                    "status": None
                    if setpoint.status.status is None
                    else setpoint.status.status.name,
                    "echoed": setpoint.status.echoed,
                    "readback": None
                    if setpoint.readback is None
                    else describe_reading(setpoint.readback),
                    "matches": setpoint.matches,
                }
            )
        return {
            "accepted": written.accepted,
            "verified": written.verified,
            "points": described,
            "operated": self._describe_operated(outstation, written.operated),
        }

    async def _der_switch(self, params: Mapping[str, Any], enable: bool) -> dict[str, Any]:
        outstation, profile, der = self._der(params)
        name = params.get("function")
        if not name or not isinstance(name, str):
            raise BadRequest("the operation names a function")
        plan = switch_plan(profile, name, enable, mode=self._mode(params))
        self._commanding(f"{'enabling' if enable else 'disabling'} a function")
        switched = await der.run(plan)
        return {
            "function": switched.function.key,
            "name": switched.function.name,
            "enable": enable,
            "accepted": switched.accepted,
            "enabled": switched.enabled,
            "status": None if switched.status is None else describe_reading(switched.status),
            "operated": self._describe_operated(outstation, switched.operated),
        }

    async def _der_enable(self, params: Mapping[str, Any]) -> dict[str, Any]:
        return await self._der_switch(params, True)

    async def _der_disable(self, params: Mapping[str, Any]) -> dict[str, Any]:
        return await self._der_switch(params, False)

    async def _der_functions(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation, profile, der = self._der(params)
        found = await der.run(functions_plan(profile))
        functions = []
        for state in found.states:
            function = state.function
            functions.append(
                {
                    "key": function.key,
                    "name": function.name,
                    "purpose": function.purpose,
                    "enable": point_address(function.enable),
                    "status": None if function.status is None else point_address(function.status),
                    "supports": point_address(function.supports),
                    "supported": state.supported,
                    "enabled": state.enabled,
                    "settings": [describe_point(profile, point) for point in function.settings],
                    "inputs": [point_address(point) for point in function.inputs],
                    "curve_settings": [point_address(point) for point in function.curve_settings],
                }
            )
        return {
            "functions": functions,
            "exchanges": [
                self._describe_exchange(outstation, exchange)
                for exchange in found.readings.exchanges
            ],
        }

    async def _der_curve(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation, profile, der = self._der(params)
        number = params.get("number")
        plan = curve_plan(profile, number, mode=self._mode(params))
        if number is not None:
            self._commanding("selecting a curve")
        selected, curve = await der.run(plan)
        return {
            "curve": self._describe_curve(outstation, curve),
            "selected": None if selected is None else self._describe_operated(outstation, selected),
        }

    async def _der_write_curve(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation, profile, der = self._der(params)
        for required in ("number", "type", "x_units", "y_units", "points"):
            if params.get(required) is None:
                raise BadRequest(f"a curve is written with {required}")
        points = params["points"]
        if isinstance(points, (str, Mapping)) or not isinstance(points, list):
            raise BadRequest("points is a list of [x, y] pairs")
        plan = write_curve_plan(
            profile,
            params["number"],
            type=params["type"],
            x_units=params["x_units"],
            y_units=params["y_units"],
            points=points,
            mode=self._mode(params),
        )
        self._commanding("writing a curve")
        written = await der.run(plan)
        return {
            "accepted": written.accepted,
            "matches": written.matches,
            "stopped_at": written.stopped_at,
            "steps": [self._describe_operated(outstation, step) for step in written.steps],
            "curve": self._describe_curve(outstation, written.readback),
        }

    def _device_profile(self, params: Mapping[str, Any]) -> str:
        """Return the Device Profile XML a request sent, or the outstation's own."""
        document = params.get("document")
        if document is None:
            outstation = self._outstation(params)
            document = self._device_profiles.get(outstation.name)
            if document is None:
                raise BadRequest(
                    f"{outstation.name} was given no Device Profile document; send one as "
                    "document, or give it with --device-profile"
                )
        if not isinstance(document, str):
            raise BadRequest("document is the Device Profile document's XML, as text")
        return document

    async def _der_document(self, params: Mapping[str, Any]) -> dict[str, Any]:
        """Lay a Device Profile document out as facts, sections and tables.

        Nothing is asked of the outstation and nothing is sent to it, so this
        needs no connection and is answered by a service that only reads.
        """
        return device_profile_view.read(self._device_profile(params)).describe()

    async def _der_compare(self, params: Mapping[str, Any]) -> dict[str, Any]:
        outstation = self._outstation(params)
        plan = compare_plan(self._device_profile(params))
        # Compared by type and index, so it needs no profile: the names are the document's.
        compared = await Der(outstation, PointMap({})).run(plan)

        def named(point_type: PointType, index: int) -> str | None:
            return self._name(outstation.name, point_type, index)

        return {
            "declared": len(compared.declared),
            "served": len(compared.served),
            "absent": [describe_declared(entry) for entry in compared.absent],
            "undeclared": [
                {"type": point_type.value, "index": index, "name": named(point_type, index)}
                for point_type, index in compared.undeclared
            ],
            "class_0": [
                {**describe_declared(entry), "carried": carried}
                for entry, carried in compared.class_0
            ],
            "points": [describe_declared(entry) for entry in compared.served],
            "exchanges": [
                {
                    "function": exchange.function.name,
                    "outcome": exchange.outcome.value,
                    "indications": iin_names(exchange.iin) if exchange.iin is not None else None,
                    "object_count": len(exchange.objects),
                    "elapsed_ms": round(exchange.elapsed * 1000, 1),
                }
                for exchange in compared.exchanges
            ],
        }

    async def _stop(self, _params: Mapping[str, Any]) -> dict[str, Any]:
        self.stopped.set()
        return {"stopping": True}

    async def close(self) -> None:
        """Close every connection, then end each connection in the capture file and close it."""
        await self.master.close()
        for recorder in self._recorders.values():
            recorder.close()
        self._recorders.clear()
        if self.capture_file is not None:
            self.capture_file.close()

operations `property` ¶

operations: tuple[str, ...]

The names of the operations, as a message's op may give them.

handle `async` ¶

handle(message: Any) -> dict[str, Any]

Answer one message. Never raises: what went wrong is the answer.

Source code in src/py1815/master/service.py

async def handle(self, message: Any) -> dict[str, Any]:
    """Answer one message. Never raises: what went wrong is the answer."""
    identifier = message.get("id") if isinstance(message, Mapping) else None
    try:
        if not isinstance(message, Mapping):
            raise BadRequest("a message is a JSON object")
        operation = self._operations.get(str(message.get("op")))
        if operation is None:
            raise BadRequest(
                f"{message.get('op')!r} is not an operation; one of "
                + ", ".join(self._operations)
            )
        params = message.get("params") or {}
        if not isinstance(params, Mapping):
            raise BadRequest("params is a JSON object")
        if "outstation" in message and "outstation" not in params:
            params = {**params, "outstation": message["outstation"]}
        _commanded.set(False)
        result = await operation(params)
        if _commanded.get():
            self._audit(str(message.get("op")), params, _outcome(result))
    except ControlNotAllowed as error:
        logger.warning(
            "dnp3 master: refused %s for %s: commanding is off",
            message.get("op") if isinstance(message, Mapping) else None,
            params.get("outstation") if isinstance(params, Mapping) else None,
        )
        return {
            "id": identifier,
            "ok": False,
            "error": {"kind": "not_allowed", "message": str(error)},
        }
    # Busy, which the association raises for a second request while one is
    # outstanding, is deliberately not caught. An outstation takes its
    # requests one at a time through a lock, so no message can cause it; if
    # it is ever raised here the fault is in this library and should be seen.
    except (ValueError, KeyError, TypeError) as error:
        text = str(error.args[0]) if isinstance(error, KeyError) and error.args else str(error)
        return {"id": identifier, "ok": False, "error": {"kind": "request", "message": text}}
    except OSError as error:
        if _commanded.get():
            self._audit(str(message.get("op")), params, f"failed: {error}")
        return {
            "id": identifier,
            "ok": False,
            "error": {"kind": "connection", "message": str(error) or type(error).__name__},
        }
    return {"id": identifier, "ok": True, "result": result}

subscribe() -> asyncio.Queue[dict[str, Any]]

A queue that receives everything that happens, until unsubscribed.

Source code in src/py1815/master/service.py

def subscribe(self) -> asyncio.Queue[dict[str, Any]]:
    """A queue that receives everything that happens, until unsubscribed."""
    queue: asyncio.Queue[dict[str, Any]] = asyncio.Queue(maxsize=SUBSCRIBER_QUEUE)
    self._subscribers.add(queue)
    return queue

publish ¶

publish(event: dict[str, Any]) -> None

Send an event to every subscriber.

A subscriber that has fallen SUBSCRIBER_QUEUE updates behind has its waiting updates replaced by one lost event that says how many were dropped, and stays subscribed. It can then reload what it shows, and memory stays bounded however long it lags.

Source code in src/py1815/master/service.py

def publish(self, event: dict[str, Any]) -> None:
    """Send an event to every subscriber.

    A subscriber that has fallen SUBSCRIBER_QUEUE updates behind has
    its waiting updates replaced by one lost event that says how many
    were dropped, and stays subscribed. It can then reload what it shows,
    and memory stays bounded however long it lags.
    """
    for queue in list(self._subscribers):
        try:
            queue.put_nowait(event)
        except asyncio.QueueFull:
            dropped = 1
            while not queue.empty():
                waiting = queue.get_nowait()
                # A lost update already waiting stands for all it dropped.
                dropped += waiting.get("dropped", 1) if waiting.get("event") == "lost" else 1
            logger.warning(
                "dnp3 master: a subscriber fell behind; dropped %d update(s)", dropped
            )
            queue.put_nowait({"event": "lost", "dropped": dropped})

set_names ¶

set_names(outstation: str, names: Names) -> None

Give an outstation's points names, for a caller that knows its map.

Source code in src/py1815/master/service.py

def set_names(self, outstation: str, names: Names) -> None:
    """Give an outstation's points names, for a caller that knows its map."""
    self._names[outstation] = {point: dict(indexed) for point, indexed in names.items()}

set_profile ¶

set_profile(outstation: str, point_map: PointMap) -> None

Give an outstation the profile it is meant to serve.

The whole map, and not only the points the outstation reports. That is what lets a caller see the difference: a point of the profile the outstation has never reported is one it does not implement, or one nobody has read yet, and a master cannot tell which from silence.

Source code in src/py1815/master/service.py

def set_profile(self, outstation: str, point_map: PointMap) -> None:
    """Give an outstation the profile it is meant to serve.

    The whole map, and not only the points the outstation reports. That
    is what lets a caller see the difference: a point of the profile the
    outstation has never reported is one it does not implement, or one
    nobody has read yet, and a master cannot tell which from silence.
    """
    self._profiles[outstation] = point_map
    self._der_profiles.pop(outstation, None)
    names: dict[PointType, dict[int, str]] = {point: {} for point in PointType}
    for (kind, index), point in point_map.points.items():
        for point_type in _KIND_TYPES[kind]:
            names[point_type][index] = point.name
    self._names[outstation] = names

set_device_profile ¶

set_device_profile(outstation: str, document: str) -> None

Give an outstation its Device Profile document.

der.compare compares what the outstation serves with it, and der.document lays it out for reading.

Raises ValueError for a document whose point lists cannot be read.

Source code in src/py1815/master/service.py

def set_device_profile(self, outstation: str, document: str) -> None:
    """Give an outstation its Device Profile document.

    ``der.compare`` compares what the outstation serves with it, and
    ``der.document`` lays it out for reading.

    Raises ValueError for a document whose point lists cannot be read.
    """
    read_device_profile(document)
    self._device_profiles[outstation] = document

attach ¶

attach(outstation: Outstation) -> None

Have what happens to an outstation published to subscribers.

Source code in src/py1815/master/service.py

def attach(self, outstation: Outstation) -> None:
    """Have what happens to an outstation published to subscribers."""
    name = outstation.name

    def on_frame(entry: Entry) -> None:
        self.publish({"event": "frame", "outstation": name, "frame": describe_entry(entry)})

    def on_exchange(exchange: Exchange) -> None:
        described = self._describe_exchange(outstation, exchange)
        self.publish({"event": "exchange", "outstation": name, "exchange": described})

    def on_unsolicited(unsolicited: Unsolicited) -> None:
        objects = [
            describe_object(decoded, self._name(name, decoded.point, decoded.index))
            for decoded in unsolicited.objects
        ]
        self.publish(
            {
                "event": "unsolicited",
                "outstation": name,
                "indications": iin_names(unsolicited.iin),
                "objects": objects,
            }
        )

    def on_connection(connected: bool) -> None:
        self.publish({"event": "connection", "outstation": name, "connected": connected})

    outstation.trace.listeners.append(on_frame)
    if self.capture_file is not None:
        recorder = Recorder(self.capture_file, outstation.trace, port=outstation.port)
        self._recorders[name] = recorder

        def on_frame_captured(entry: Entry) -> None:
            # A listener's error reaches the code that recorded the frame,
            # which is the receive loop or a request. A full disk must not
            # end the DNP3 connection, so the capture stops instead.
            try:
                recorder.record(entry)
            except (OSError, ValueError):
                logger.exception("dnp3 master: writing the capture of %s failed", name)
                self._stop_capture()

        self._capture_listeners[name] = on_frame_captured
        outstation.trace.listeners.append(on_frame_captured)
    outstation.on_exchange = on_exchange
    outstation.on_unsolicited = on_unsolicited
    outstation.on_connection = on_connection

close `async` ¶

close() -> None

Close every connection, then end each connection in the capture file and close it.

Source code in src/py1815/master/service.py

async def close(self) -> None:
    """Close every connection, then end each connection in the capture file and close it."""
    await self.master.close()
    for recorder in self._recorders.values():
        recorder.close()
    self._recorders.clear()
    if self.capture_file is not None:
        self.capture_file.close()

LineServer ¶

The service a line of JSON at a time, over a local TCP socket.

For a test rig. A request is one JSON object on one line, and so is its response. A connection that sends {"op": "subscribe"} is also sent what happens unasked, each as a line with an event field.

Source code in src/py1815/master/service.py

class LineServer:
    """The service a line of JSON at a time, over a local TCP socket.

    For a test rig. A request is one JSON object on one line, and so is its
    response. A connection that sends ``{"op": "subscribe"}`` is also sent
    what happens unasked, each as a line with an ``event`` field.
    """

    def __init__(self, service: Service, *, bind: str = "127.0.0.1:0") -> None:
        self._service = service
        self._host, self._port = _split_bind(bind)
        if not _is_loopback(self._host):
            raise ValueError(
                f"the line service listens on this machine only, and {self._host} is not it"
            )
        self._server: asyncio.Server | None = None

    @property
    def port(self) -> int:
        assert self._server is not None
        return int(self._server.sockets[0].getsockname()[1])

    async def start(self) -> None:
        # A line may be as long as a message over HTTP may be, which is what a
        # Device Profile document sent in one needs. The stream's own default
        # is far shorter.
        self._server = await asyncio.start_server(
            self._handle, self._host, self._port, limit=_MAX_REQUEST_BODY
        )

    async def stop(self) -> None:
        if self._server is not None:
            self._server.close()
            await self._server.wait_closed()
            self._server = None

    async def _handle(self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
        writing = asyncio.Lock()
        forwarding: asyncio.Task[None] | None = None
        queue: asyncio.Queue[dict[str, Any]] | None = None

        async def send(message: dict[str, Any]) -> None:
            async with writing:
                writer.write(json.dumps(message, separators=(",", ":")).encode() + b"\n")
                await writer.drain()

        async def forward(events: asyncio.Queue[dict[str, Any]]) -> None:
            while True:
                await send(await events.get())

        try:
            while True:
                try:
                    line = await reader.readline()
                except ValueError:
                    # The line is longer than the limit, and the rest of it is
                    # still to come, so the connection cannot be read on.
                    megabytes = _MAX_REQUEST_BODY >> 20
                    await send(
                        {
                            "id": None,
                            "ok": False,
                            "error": {
                                "kind": "request",
                                "message": f"a line is at most {megabytes} MiB; "
                                "the connection is closed",
                            },
                        }
                    )
                    return
                if not line:
                    return
                if not line.strip():
                    continue
                try:
                    message = json.loads(line)
                except ValueError:
                    await send(
                        {
                            "id": None,
                            "ok": False,
                            "error": {"kind": "request", "message": "a line is one JSON object"},
                        }
                    )
                    continue
                if isinstance(message, dict) and message.get("op") == "subscribe":
                    if queue is None:
                        queue = self._service.subscribe()
                        forwarding = asyncio.create_task(forward(queue))
                    await send(
                        {"id": message.get("id"), "ok": True, "result": {"subscribed": True}}
                    )
                    continue
                await send(await self._service.handle(message))
        except (OSError, asyncio.IncompleteReadError):
            return
        finally:
            if forwarding is not None:
                forwarding.cancel()
            if queue is not None:
                self._service.unsubscribe(queue)
            writer.close()
            with contextlib.suppress(OSError):
                await writer.wait_closed()

HttpServer ¶

The service over HTTP, and the console's files.

POST /api/<operation> takes an operation's parameters and returns its answer, and POST /api takes a whole message that names its operation. GET /events is a stream of server-sent events. Everything else is a file of the console, among them /openapi.json, which describes the routes.

It listens on this machine unless told otherwise, and told otherwise it requires a token of every request to the service. A request is refused unless it comes from a page this server served: a browser will carry a request from any page to a local port, and a tool pointed at real equipment must not take one.

Source code in src/py1815/master/service.py

class HttpServer:
    """The service over HTTP, and the console's files.

    ``POST /api/<operation>`` takes an operation's parameters and returns its
    answer, and ``POST /api`` takes a whole message that names its operation.
    ``GET /events`` is a stream of server-sent events. Everything else is a
    file of the console, among them ``/openapi.json``, which describes the
    routes.

    It listens on this machine unless told otherwise, and told otherwise it
    requires a token of every request to the service. A request is refused
    unless it comes
    from a page this server served: a browser will carry a request from any
    page to a local port, and a tool pointed at real equipment must not take
    one.
    """

    def __init__(
        self,
        service: Service,
        *,
        bind: str = DEFAULT_HTTP_BIND,
        token: str | None = None,
        without_token: bool = False,
        directory: pathlib.Path = CONSOLE_DIRECTORY,
    ) -> None:
        """
        Args:
            service: The service to serve.
            bind: The address and port to listen on.
            token: Required of every request to the service, when given.
            without_token: Listen beyond this machine with no token. For a
                container, whose own network is not the one it is published
                on: there, what decides who can reach the port is how it was
                published, and the caller is saying that it was published
                narrowly. A request still has to name this machine as its
                host, so a page elsewhere cannot be pointed at it.
            directory: Where the console's files are.
        """
        self._service = service
        self._host, self._port = _split_bind(bind)
        if not _is_loopback(self._host) and not token and not without_token:
            raise ValueError(
                f"listening on {self._host} is listening to other machines, and needs a token"
            )
        self._token = token
        self._directory = directory
        self._server: asyncio.Server | None = None
        self._streams: set[asyncio.Task[None]] = set()

    @property
    def port(self) -> int:
        assert self._server is not None
        return int(self._server.sockets[0].getsockname()[1])

    @property
    def url(self) -> str:
        """Where to open the console, with the token when there is one.

        An address that means "every interface" is not one a browser can go
        to, so this machine's own name stands in for it.
        """
        host = "localhost" if self._host in ("0.0.0.0", "::") else self._host
        host = f"[{host}]" if ":" in host else host
        suffix = f"/?token={self._token}" if self._token else "/"
        return f"http://{host}:{self.port}{suffix}"

    async def start(self) -> None:
        self._server = await asyncio.start_server(self._handle, self._host, self._port)

    async def stop(self) -> None:
        if self._server is not None:
            self._server.close()
            for stream in list(self._streams):
                stream.cancel()
            await self._server.wait_closed()
            self._server = None

    # ------------------------------------------------------------- requests

    async def _handle(self, reader: asyncio.StreamReader, writer: asyncio.StreamWriter) -> None:
        task = asyncio.current_task()
        try:
            await self._serve(reader, writer, task)
        except (OSError, asyncio.IncompleteReadError, asyncio.LimitOverrunError, ValueError):
            pass
        except asyncio.CancelledError:
            pass
        finally:
            if task is not None:
                self._streams.discard(task)
            writer.close()
            with contextlib.suppress(OSError):
                await writer.wait_closed()

    async def _serve(
        self,
        reader: asyncio.StreamReader,
        writer: asyncio.StreamWriter,
        task: asyncio.Task[None] | None,
    ) -> None:
        head = await reader.readuntil(b"\r\n\r\n")
        if len(head) > _MAX_HEADER:
            return await self._respond(writer, 400, "the request header is too large")
        lines = head.decode("latin-1").split("\r\n")
        try:
            method, target, _version = lines[0].split(" ")
        except ValueError:
            return await self._respond(writer, 400, "that is not an HTTP request")
        headers = {}
        for line in lines[1:]:
            name, separator, value = line.partition(":")
            if separator:
                headers[name.strip().lower()] = value.strip()
        url = urlsplit(target)
        query = parse_qs(url.query)

        refusal = self._refusal(headers, query, url.path)
        if refusal is not None:
            return await self._respond(writer, *refusal)

        if url.path == "/api" or url.path.startswith("/api/"):
            if method != "POST":
                return await self._respond(writer, 405, "the service is asked with POST")
            if headers.get("content-type", "").split(";")[0].strip() != "application/json":
                return await self._respond(writer, 400, "a message is sent as application/json")
            length = int(headers.get("content-length", "0"))
            if length > _MAX_REQUEST_BODY:
                return await self._respond(writer, 413, "the message is too large")
            body = await reader.readexactly(length)
            try:
                sent = json.loads(body) if body.strip() else None
            except ValueError:
                return await self._respond(writer, 400, "the message is not JSON")
            if url.path == "/api":
                # One message, naming its operation: the form the line service takes.
                message = sent
            else:
                # A route for each operation, whose body is its parameters.
                operation = url.path.removeprefix("/api/")
                if operation not in self._service.operations:
                    return await self._respond(writer, 404, "there is no such operation")
                if sent is not None and not isinstance(sent, dict):
                    return await self._respond(writer, 400, "parameters are a JSON object")
                message = {"op": operation, "params": sent or {}}
            answer = await self._service.handle(message)
            return await self._respond_json(writer, answer)

        if method != "GET":
            return await self._respond(writer, 405, "that is read with GET")
        if url.path == "/events":
            if task is not None:
                self._streams.add(task)
            return await self._events(writer)
        return await self._file(writer, url.path)

    def _refusal(
        self, headers: Mapping[str, str], query: Mapping[str, list[str]], path: str
    ) -> tuple[int, str] | None:
        host = headers.get("host", "")
        origin = headers.get("origin")
        if origin is not None and urlsplit(origin).netloc != host:
            return 403, "the request comes from another site"
        if self._token is None:
            # Listening on this machine only. A name that resolves here but is
            # not this machine's own is how another site reaches a local port.
            name = host.rsplit(":", 1)[0].strip("[]")
            if not _is_loopback(name):
                return 403, "the request names a host this server is not"
            return None
        if not (path == "/api" or path.startswith("/api/") or path == "/events"):
            # The console's own files. They are the same for everyone and say
            # nothing about any outstation, and a page cannot put a token on
            # the stylesheet and script it links to. What the token guards is
            # the service: every operation, and the stream of what happens.
            return None
        offered = headers.get("authorization", "").removeprefix("Bearer ").strip()
        offered = offered or (query.get("token") or [""])[0]
        if not secrets.compare_digest(offered.encode(), self._token.encode()):
            return 401, "the token is missing or wrong"
        return None

    async def _respond(self, writer: asyncio.StreamWriter, status: int, text: str) -> None:
        await self._send(writer, status, "text/plain; charset=utf-8", text.encode())

    async def _respond_json(self, writer: asyncio.StreamWriter, payload: Any) -> None:
        body = json.dumps(payload, separators=(",", ":")).encode()
        await self._send(writer, 200, "application/json", body)

    @staticmethod
    async def _send(
        writer: asyncio.StreamWriter, status: int, content_type: str, body: bytes
    ) -> None:
        head = (
            f"HTTP/1.1 {status} {_REASONS.get(status, 'Error')}\r\n"
            f"Content-Type: {content_type}\r\n"
            f"Content-Length: {len(body)}\r\n"
            "Cache-Control: no-store\r\n"
            "X-Content-Type-Options: nosniff\r\n"
            "Connection: close\r\n\r\n"
        )
        writer.write(head.encode("latin-1") + body)
        await writer.drain()

    async def _file(self, writer: asyncio.StreamWriter, path: str) -> None:
        name = "index.html" if path in ("", "/") else path.lstrip("/")
        # Only a file that is in the console's own directory, by its own name:
        # nothing in the path is followed.
        served = {entry.name: entry for entry in self._directory.iterdir() if entry.is_file()}
        entry = served.get(name)
        if entry is None:
            return await self._respond(writer, 404, "there is no such file")
        content_type = mimetypes.guess_type(entry.name)[0] or "application/octet-stream"
        if content_type.startswith("text/") or content_type.endswith("javascript"):
            content_type += "; charset=utf-8"
        await self._send(writer, 200, content_type, entry.read_bytes())

    async def _events(self, writer: asyncio.StreamWriter) -> None:
        queue = self._service.subscribe()
        try:
            writer.write(
                b"HTTP/1.1 200 OK\r\n"
                b"Content-Type: text/event-stream\r\n"
                b"Cache-Control: no-store\r\n"
                b"Connection: close\r\n\r\n"
                b": connected\n\n"
            )
            await writer.drain()
            while True:
                try:
                    event = await asyncio.wait_for(queue.get(), 15.0)
                except TimeoutError:
                    # A comment, so a connection nobody is writing to is seen
                    # to have gone away.
                    writer.write(b": waiting\n\n")
                else:
                    payload = json.dumps(event, separators=(",", ":"))
                    writer.write(f"data: {payload}\n\n".encode())
                try:
                    await asyncio.wait_for(writer.drain(), STREAM_WRITE_TIMEOUT)
                except TimeoutError:
                    # The browser stopped reading. Close the stream: an
                    # EventSource opens a new one, and the console reloads.
                    logger.warning(
                        "dnp3 master: closed an event stream that stopped reading for %g s",
                        STREAM_WRITE_TIMEOUT,
                    )
                    return
        finally:
            self._service.unsubscribe(queue)

url `property` ¶

url: str

Where to open the console, with the token when there is one.

An address that means "every interface" is not one a browser can go to, so this machine's own name stands in for it.

split_enumeration ¶

split_enumeration(
    name: str,
) -> tuple[str, list[dict[str, str]] | None]

A point's name without the values it lists, and the values apart.

The profile's tables write an enumerated point's values into its name: Curve Type. Enumeration: <0> Curve is not defined <2> Volt-Var. That is the right thing to have and the wrong thing to print in a column, so this takes it apart: ("Curve Type", [{"value": "0", "name": "Curve is not defined"}, ...]). A value may be a number, a range (11-255) or a number and up (99+), as the tables have them.

A name that lists fewer than two values is returned whole, with None. A value written in passing, as in Default is <3>, names nothing after it and is not an entry: it is left in the name, without its brackets.

Source code in src/py1815/master/service.py

def split_enumeration(name: str) -> tuple[str, list[dict[str, str]] | None]:
    """A point's name without the values it lists, and the values apart.

    The profile's tables write an enumerated point's values into its name:
    ``Curve Type. Enumeration: <0> Curve is not defined <2> Volt-Var``. That
    is the right thing to have and the wrong thing to print in a column, so
    this takes it apart: ``("Curve Type", [{"value": "0", "name": "Curve is
    not defined"}, ...])``. A value may be a number, a range (``11-255``) or
    a number and up (``99+``), as the tables have them.

    A name that lists fewer than two values is returned whole, with None. A
    value written in passing, as in ``Default is <3>``, names nothing after
    it and is not an entry: it is left in the name, without its brackets.
    """
    found = list(_ENUMERATED.finditer(name))
    entries: list[tuple[int, str, str]] = []
    for position, token in enumerate(found):
        end = found[position + 1].start() if position + 1 < len(found) else len(name)
        text = name[token.end() : end].strip().lstrip("=:-").strip()
        if text.strip(" .,;:"):
            entries.append((token.start(), "".join(token.group(1).split()), text))
    if len(entries) < 2:
        return name, None
    label = _ENUMERATED.sub(lambda token: token.group(1), name[: entries[0][0]])
    label = _ENUMERATION_LEAD.sub("", label).strip()
    values = [{"value": value, "name": text.rstrip(" .,;")} for _start, value, text in entries]
    return label or name, values

flag_names ¶

flag_names(
    point: PointType | None, flags: int | None
) -> list[str] | None

A flag octet as the names of the bits set, or None where there is none.

Source code in src/py1815/master/service.py

def flag_names(point: PointType | None, flags: int | None) -> list[str] | None:
    """A flag octet as the names of the bits set, or None where there is none."""
    if flags is None:
        return None
    if point is None:
        return [f"0x{flags:02X}"] if flags else []
    return [
        bit.name
        for bit in _FLAGS[point]
        if bit.name is not None and bit.name != "STATE" and flags & bit
    ]

describe_point ¶

describe_point(
    profile: DerProfile, point: Point
) -> dict[str, Any]

Describe a point of the profile: where it is, its name, its units and its range.

Source code in src/py1815/master/service.py

def describe_point(profile: DerProfile, point: Point) -> dict[str, Any]:
    """Describe a point of the profile: where it is, its name, its units and its range."""
    low, high = DerProfile.limits(point) if point.kind.is_analog else (None, None)
    mirror = profile.mirror(point) if point.kind.is_output else None
    return {
        "address": point_address(point),
        "type": _KIND_TYPES[point.kind][0].value,
        "index": point.index,
        "name": point.name,
        "label": split_enumeration(point_label(point))[0],
        "units": _units(point),
        "multiplier": point.multiplier,
        "minimum": _number(low),
        "maximum": _number(high),
        "states": list(point.states) if point.states is not None else None,
        "mirror": None if mirror is None else point_address(mirror),
    }

describe_reading ¶

describe_reading(reading: Reading) -> dict[str, Any]

Describe one point as an outstation reported it, in engineering units.

Source code in src/py1815/master/service.py

def describe_reading(reading: Reading) -> dict[str, Any]:
    """Describe one point as an outstation reported it, in engineering units."""
    point = reading.point
    point_type = _KIND_TYPES[point.kind][0]
    return {
        "address": point_address(point),
        "type": point_type.value,
        "index": point.index,
        "name": point.name,
        "label": split_enumeration(point_label(point))[0],
        "units": _units(point),
        "value": _number(reading.value),
        "raw": _number(reading.raw),
        "state": reading.state,
        "flags": flag_names(point_type, reading.flags),
        "quality": reading.quality,
        "time_ms": reading.time_ms,
    }

describe_check ¶

describe_check(check: Check) -> dict[str, Any]

Return a check of the catalog as a JSON-compatible dict.

Source code in src/py1815/master/service.py

def describe_check(check: Check) -> dict[str, Any]:
    """Return a check of the catalog as a JSON-compatible dict."""
    return {
        "id": check.id,
        "title": check.title,
        "set": check.group,
        "writes": check.commands,
        "deviates": check.deviates,
    }

Configuration¶

py1815.master.config ¶

The master's configuration: every setting in one JSON document.

A configuration has its top-level settings, an evaluate object with the settings of py1815-master evaluate, a defaults object with the settings shared by every outstation, and a list of outstations. An outstation entry needs a name and a host; any other setting it gives overrides the default.

.. code-block:: json

{
  "allow_control": false,
  "defaults": {"reconnect": 5, "repeat": {"integrity": 30}},
  "outstations": [
    {"name": "lab", "host": "192.0.2.10"},
    {"name": "bench", "host": "192.0.2.11", "port": 20001, "manual": true}
  ]
}

py1815-master config prints the complete document, with every default filled in, for editing. console, serve and evaluate load one with --config FILE.

Unknown settings and values of the wrong type are errors that name where they are, so a typing mistake is reported instead of ignored.

ConfigError ¶

Bases: ValueError

A configuration that cannot be used. The message says where and why.

Source code in src/py1815/settings.py

class ConfigError(ValueError):
    """A configuration that cannot be used. The message says where and why."""

OutstationConfig `dataclass` ¶

The settings of one outstation. name and host are required.

Source code in src/py1815/master/config.py

@dataclass(frozen=True)
class OutstationConfig:
    """The settings of one outstation. ``name`` and ``host`` are required."""

    name: str = ""
    host: str = ""
    port: int = 20000
    outstation_address: int = 1024
    master_address: int = 1
    #: Seconds to wait for a response, and for each further fragment of one.
    response_timeout: float = DEFAULT_RESPONSE_TIMEOUT
    #: Times a read that times out is sent again. No other request is.
    read_retries: int = DEFAULT_READ_RETRIES
    #: Seconds to wait for the TCP connection.
    connect_timeout: float = DEFAULT_CONNECT_TIMEOUT
    #: Connect over TLS with these files, or None for plain TCP.
    tls: TlsSettings | None = None
    #: Connect when the master starts.
    connect: bool = True
    #: Seconds between reconnection attempts after a lost connection, or None
    #: for no reconnection.
    reconnect: float | None = DEFAULT_RECONNECT
    #: Confirm response fragments that ask for confirmation.
    confirm: bool = True
    #: Send only what is asked for. Disables every task and confirmation,
    #: whatever ``tasks`` and ``confirm`` say.
    manual: bool = False
    #: The outstation is an IEEE 1815.2 DER: name its points from the profile
    #: tables.
    profile: bool = False
    #: Path of the outstation's DNP3 Device Profile document, which
    #: ``der.compare`` compares what it serves with and ``der.document``
    #: lays out for reading. None for none.
    device_profile: str | None = None
    #: The automatic tasks. The two that write (``clear_restart`` and
    #: ``write_time``) only run when ``allow_control`` is on.
    tasks: Tasks = field(default_factory=Tasks)
    #: Seconds between repeated scans, by kind. None means not repeated.
    #: ``outputs`` may be ``"with_integrity"``.
    repeat: Mapping[str, float | str | None] = field(
        default_factory=lambda: {"integrity": None, "events": None, "outputs": WITH_INTEGRITY}
    )

    #: Settings that take a number of seconds greater than zero.
    _SECONDS = ("response_timeout", "connect_timeout")
    _BOOLEANS = ("connect", "confirm", "manual", "profile")

    def changed(self, given: Mapping[str, Any], where: str) -> OutstationConfig:
        """Return a copy with the settings in ``given`` applied.

        ``tasks``, ``repeat`` and ``tls`` are merged: a setting they leave out
        keeps its current value. ``tls`` given as null is plain TCP.
        """
        allowed = [each.name for each in fields(self)]
        settings.section(given, where, allowed)
        values: dict[str, Any] = {each.name: getattr(self, each.name) for each in fields(self)}
        for key, value in given.items():
            at = f"{where}.{key}"
            if key in ("name", "host"):
                values[key] = settings.text(value, at)
            elif key == "port":
                values[key] = settings.integer(value, at, 1, 65535)
            elif key in ("outstation_address", "master_address"):
                values[key] = settings.integer(value, at, 0, settings.MAX_ADDRESS)
            elif key == "read_retries":
                values[key] = settings.integer(value, at, 0, _MAX_READ_RETRIES)
            elif key == "tls":
                values[key] = self._tls(value, at)
            elif key in self._SECONDS:
                values[key] = settings.seconds(value, at)
            elif key in self._BOOLEANS:
                values[key] = settings.boolean(value, at)
            elif key == "reconnect":
                values[key] = None if value is None else settings.seconds(value, at)
            elif key == "device_profile":
                values[key] = None if value is None else settings.text(value, at)
            elif key == "tasks":
                if not isinstance(value, Mapping):
                    raise ConfigError(f"{at} must be an object")
                try:
                    values[key] = self.tasks.changed(value)
                except ValueError as error:
                    raise ConfigError(f"{at}: {error}") from None
            else:
                values[key] = self._repeat(value, at)
        if values["outstation_address"] == values["master_address"]:
            raise ConfigError(f"{where}: outstation_address and master_address must be different")
        return OutstationConfig(**values)

    def _tls(self, given: Any, where: str) -> TlsSettings | None:
        """Return the TLS settings, merged with these as ``tasks`` and ``repeat`` are."""
        if given is None:
            return None
        if not isinstance(given, Mapping):
            raise ConfigError(f"{where} must be an object, or null for plain TCP")
        merged = {**(self.tls.describe() if self.tls is not None else {}), **given}
        try:
            return TlsSettings.from_mapping(merged)
        except ValueError as error:
            raise ConfigError(f"{where}: {error}") from None

    def _repeat(self, given: Any, where: str) -> dict[str, float | str | None]:
        merged = dict(self.repeat)
        for kind, value in settings.section(given, where, REPEATED).items():
            if value is None or (kind == requests.OUTPUTS and value == WITH_INTEGRITY):
                merged[kind] = value
            else:
                merged[kind] = settings.seconds(value, f"{where}.{kind}")
        return merged

    def output_interval(self) -> float | None:
        """Seconds between reads of output status, with ``with_integrity`` resolved."""
        outputs = self.repeat[requests.OUTPUTS]
        if outputs == WITH_INTEGRITY:
            outputs = self.repeat["integrity"]
        return None if outputs is None else float(outputs)

    def add_params(self, *, allow_control: bool) -> dict[str, Any]:
        """Return the parameters of the service's ``add`` operation for this outstation."""
        integrity, events = self.repeat["integrity"], self.repeat["events"]
        params: dict[str, Any] = {
            "name": self.name,
            "host": self.host,
            "port": self.port,
            "outstation_address": self.outstation_address,
            "master_address": self.master_address,
            "response_timeout": self.response_timeout,
            "read_retries": self.read_retries,
            "connect_timeout": self.connect_timeout,
            "tls": None if self.tls is None else self.tls.describe(),
            "connect": self.connect,
            "reconnect": self.reconnect,
            "integrity_interval": integrity,
            "event_interval": events,
            "output_interval": self.output_interval(),
        }
        if self.manual:
            params["manual"] = True
            return params
        tasks = self.tasks.describe()
        if not allow_control:
            # A read-only service refuses a request for a task that writes, so
            # leave those two to its default, which is off.
            for name in WRITING:
                del tasks[name]
        params["confirm"] = self.confirm
        params["tasks"] = tasks
        return params

    def describe(self) -> dict[str, Any]:
        """Return every setting as a JSON-compatible dict."""
        described: dict[str, Any] = {}
        for each in fields(self):
            value = getattr(self, each.name)
            if each.name in ("tasks", "tls") and value is not None:
                value = value.describe()
            elif each.name == "repeat":
                value = dict(value)
            described[each.name] = value
        return described

changed ¶

changed(
    given: Mapping[str, Any], where: str
) -> OutstationConfig

Return a copy with the settings in given applied.

tasks, repeat and tls are merged: a setting they leave out keeps its current value. tls given as null is plain TCP.

Source code in src/py1815/master/config.py

def changed(self, given: Mapping[str, Any], where: str) -> OutstationConfig:
    """Return a copy with the settings in ``given`` applied.

    ``tasks``, ``repeat`` and ``tls`` are merged: a setting they leave out
    keeps its current value. ``tls`` given as null is plain TCP.
    """
    allowed = [each.name for each in fields(self)]
    settings.section(given, where, allowed)
    values: dict[str, Any] = {each.name: getattr(self, each.name) for each in fields(self)}
    for key, value in given.items():
        at = f"{where}.{key}"
        if key in ("name", "host"):
            values[key] = settings.text(value, at)
        elif key == "port":
            values[key] = settings.integer(value, at, 1, 65535)
        elif key in ("outstation_address", "master_address"):
            values[key] = settings.integer(value, at, 0, settings.MAX_ADDRESS)
        elif key == "read_retries":
            values[key] = settings.integer(value, at, 0, _MAX_READ_RETRIES)
        elif key == "tls":
            values[key] = self._tls(value, at)
        elif key in self._SECONDS:
            values[key] = settings.seconds(value, at)
        elif key in self._BOOLEANS:
            values[key] = settings.boolean(value, at)
        elif key == "reconnect":
            values[key] = None if value is None else settings.seconds(value, at)
        elif key == "device_profile":
            values[key] = None if value is None else settings.text(value, at)
        elif key == "tasks":
            if not isinstance(value, Mapping):
                raise ConfigError(f"{at} must be an object")
            try:
                values[key] = self.tasks.changed(value)
            except ValueError as error:
                raise ConfigError(f"{at}: {error}") from None
        else:
            values[key] = self._repeat(value, at)
    if values["outstation_address"] == values["master_address"]:
        raise ConfigError(f"{where}: outstation_address and master_address must be different")
    return OutstationConfig(**values)

output_interval ¶

output_interval() -> float | None

Seconds between reads of output status, with with_integrity resolved.

Source code in src/py1815/master/config.py

def output_interval(self) -> float | None:
    """Seconds between reads of output status, with ``with_integrity`` resolved."""
    outputs = self.repeat[requests.OUTPUTS]
    if outputs == WITH_INTEGRITY:
        outputs = self.repeat["integrity"]
    return None if outputs is None else float(outputs)

add_params ¶

add_params(*, allow_control: bool) -> dict[str, Any]

Return the parameters of the service's add operation for this outstation.

Source code in src/py1815/master/config.py

def add_params(self, *, allow_control: bool) -> dict[str, Any]:
    """Return the parameters of the service's ``add`` operation for this outstation."""
    integrity, events = self.repeat["integrity"], self.repeat["events"]
    params: dict[str, Any] = {
        "name": self.name,
        "host": self.host,
        "port": self.port,
        "outstation_address": self.outstation_address,
        "master_address": self.master_address,
        "response_timeout": self.response_timeout,
        "read_retries": self.read_retries,
        "connect_timeout": self.connect_timeout,
        "tls": None if self.tls is None else self.tls.describe(),
        "connect": self.connect,
        "reconnect": self.reconnect,
        "integrity_interval": integrity,
        "event_interval": events,
        "output_interval": self.output_interval(),
    }
    if self.manual:
        params["manual"] = True
        return params
    tasks = self.tasks.describe()
    if not allow_control:
        # A read-only service refuses a request for a task that writes, so
        # leave those two to its default, which is off.
        for name in WRITING:
            del tasks[name]
    params["confirm"] = self.confirm
    params["tasks"] = tasks
    return params

describe ¶

describe() -> dict[str, Any]

Return every setting as a JSON-compatible dict.

Source code in src/py1815/master/config.py

def describe(self) -> dict[str, Any]:
    """Return every setting as a JSON-compatible dict."""
    described: dict[str, Any] = {}
    for each in fields(self):
        value = getattr(self, each.name)
        if each.name in ("tasks", "tls") and value is not None:
            value = value.describe()
        elif each.name == "repeat":
            value = dict(value)
        described[each.name] = value
    return described

EvaluateConfig `dataclass` ¶

The settings of py1815-master evaluate.

Source code in src/py1815/master/config.py

@dataclass(frozen=True)
class EvaluateConfig:
    """The settings of ``py1815-master evaluate``."""

    #: Seconds to wait for the outstation to reach a state it was commanded to.
    settle: float = DEFAULT_SETTLE
    #: How many curves the outstation stores, or None to find out by
    #: selecting each in turn.
    curves: int | None = None
    #: The checks to run, by identifier or set, or None for all of them.
    checks: tuple[str, ...] | None = None
    #: Path to write the report to as JSON, or None for no file.
    report: str | None = None
    #: Run the checks that have the master break a protocol rule on purpose.
    allow_deviations: bool = False

    @classmethod
    def from_mapping(cls, given: Any, where: str = "evaluate") -> EvaluateConfig:
        """Build the settings from a decoded JSON object."""
        allowed = [each.name for each in fields(cls)]
        values: dict[str, Any] = {}
        for key, value in settings.section(given, where, allowed).items():
            at = f"{where}.{key}"
            if key == "settle":
                values[key] = settings.seconds(value, at)
            elif key == "curves":
                values[key] = None if value is None else settings.integer(value, at, 1, 65535)
            elif key == "report":
                values[key] = None if value is None else settings.text(value, at)
            elif key == "allow_deviations":
                values[key] = settings.boolean(value, at)
            elif value is not None:
                if isinstance(value, (str, bytes, Mapping)) or not isinstance(value, Sequence):
                    raise ConfigError(f"{at} must be a list of check names, or null for all")
                values[key] = tuple(
                    settings.text(each, f"{at}[{position}]") for position, each in enumerate(value)
                )
        return cls(**values)

    def describe(self) -> dict[str, Any]:
        """Return every setting as a JSON-compatible dict."""
        return {
            "settle": self.settle,
            "curves": self.curves,
            "checks": None if self.checks is None else list(self.checks),
            "report": self.report,
            "allow_deviations": self.allow_deviations,
        }

from_mapping `classmethod` ¶

from_mapping(
    given: Any, where: str = "evaluate"
) -> EvaluateConfig

Build the settings from a decoded JSON object.

Source code in src/py1815/master/config.py

@classmethod
def from_mapping(cls, given: Any, where: str = "evaluate") -> EvaluateConfig:
    """Build the settings from a decoded JSON object."""
    allowed = [each.name for each in fields(cls)]
    values: dict[str, Any] = {}
    for key, value in settings.section(given, where, allowed).items():
        at = f"{where}.{key}"
        if key == "settle":
            values[key] = settings.seconds(value, at)
        elif key == "curves":
            values[key] = None if value is None else settings.integer(value, at, 1, 65535)
        elif key == "report":
            values[key] = None if value is None else settings.text(value, at)
        elif key == "allow_deviations":
            values[key] = settings.boolean(value, at)
        elif value is not None:
            if isinstance(value, (str, bytes, Mapping)) or not isinstance(value, Sequence):
                raise ConfigError(f"{at} must be a list of check names, or null for all")
            values[key] = tuple(
                settings.text(each, f"{at}[{position}]") for position, each in enumerate(value)
            )
    return cls(**values)

describe ¶

describe() -> dict[str, Any]

Return every setting as a JSON-compatible dict.

Source code in src/py1815/master/config.py

def describe(self) -> dict[str, Any]:
    """Return every setting as a JSON-compatible dict."""
    return {
        "settle": self.settle,
        "curves": self.curves,
        "checks": None if self.checks is None else list(self.checks),
        "report": self.report,
        "allow_deviations": self.allow_deviations,
    }

MasterConfig `dataclass` ¶

The master's whole configuration.

Source code in src/py1815/master/config.py

@dataclass(frozen=True)
class MasterConfig:
    """The master's whole configuration."""

    #: Allow the operations and tasks that write to an outstation.
    allow_control: bool = False
    #: Address and port to listen on, or None for the command's default.
    bind: str | None = None
    #: Path of the profile tables file, or None for the usual locations.
    tables: str | None = None
    #: Seconds to keep trying to reach an outstation that is not there at
    #: startup. Zero tries once.
    connect_wait: float = 0.0
    #: Path of a pcap file to write every frame to as it is sent or received,
    #: or None for no file.
    capture: str | None = None
    #: Start a new capture file once the current one reaches this many
    #: megabytes, keeping ``capture_keep`` older files.
    capture_max_mb: float = 100.0
    capture_keep: int = 10
    #: Path of a log file, or None to log to the terminal only. The file is
    #: rotated at ``log_max_mb`` megabytes, keeping ``log_keep`` older files.
    log_file: str | None = None
    log_max_mb: float = 10.0
    log_keep: int = 5
    #: The lowest level written to the log file: debug, info or warning.
    log_level: str = "info"
    #: The settings of ``py1815-master evaluate``.
    evaluate: EvaluateConfig = field(default_factory=EvaluateConfig)
    #: The settings an outstation has unless its own entry says otherwise.
    defaults: OutstationConfig = field(default_factory=OutstationConfig)
    outstations: tuple[OutstationConfig, ...] = ()

    @classmethod
    def from_mapping(cls, given: Mapping[str, Any]) -> MasterConfig:
        """Build a configuration from a decoded JSON document.

        Raises :class:`ConfigError` for an unknown setting, a value of the
        wrong type, an outstation with no name or host, or a name used twice.
        """
        settings.section(given, "configuration", _TOP_LEVEL)
        values: dict[str, Any] = {}
        if "allow_control" in given:
            values["allow_control"] = settings.boolean(given["allow_control"], "allow_control")
        for key in ("bind", "tables", "capture", "log_file"):
            if given.get(key) is not None:
                values[key] = settings.text(given[key], key)
        for key in ("capture_max_mb", "log_max_mb"):
            if key in given:
                values[key] = settings.seconds(given[key], key, unit="megabytes")
        if "capture_keep" in given:
            values["capture_keep"] = settings.integer(
                given["capture_keep"], "capture_keep", 0, 1000
            )
        if "log_keep" in given:
            # At least 1: Python's rotating log handler does not rotate with 0,
            # and the file would then grow without limit.
            values["log_keep"] = settings.integer(given["log_keep"], "log_keep", 1, 1000)
        if "log_level" in given:
            level = given["log_level"]
            if level not in LOG_LEVELS:
                raise ConfigError(f"log_level must be one of {', '.join(LOG_LEVELS)}")
            values["log_level"] = level
        if "connect_wait" in given:
            wait = given["connect_wait"]
            if (
                isinstance(wait, bool)
                or not isinstance(wait, (int, float))
                or not math.isfinite(wait)
                or wait < 0
            ):
                raise ConfigError("connect_wait must be a number of seconds, zero or more")
            values["connect_wait"] = float(wait)
        if "evaluate" in given:
            values["evaluate"] = EvaluateConfig.from_mapping(given["evaluate"])

        shared = given.get("defaults", {})
        for required in ("name", "host"):
            if isinstance(shared, Mapping) and required in shared:
                raise ConfigError(f"defaults.{required} belongs in an outstation's own entry")
        defaults = OutstationConfig().changed(shared, "defaults")
        values["defaults"] = defaults

        entries = given.get("outstations", [])
        if isinstance(entries, (str, bytes, Mapping)) or not isinstance(entries, Sequence):
            raise ConfigError("outstations must be a list")
        outstations: list[OutstationConfig] = []
        for position, entry in enumerate(entries):
            where = f"outstations[{position}]"
            outstation = defaults.changed(entry, where)
            for required in ("name", "host"):
                if not getattr(outstation, required):
                    raise ConfigError(f"{where}.{required} is required")
            if any(outstation.name == other.name for other in outstations):
                raise ConfigError(f"{where}.name {outstation.name!r} is used more than once")
            outstations.append(outstation)
        values["outstations"] = tuple(outstations)
        return cls(**values)

    @classmethod
    def load(cls, path: str | pathlib.Path) -> MasterConfig:
        """Read a configuration from a JSON file."""
        return cls.from_mapping(read(path))

    def describe(self) -> dict[str, Any]:
        """Return the complete configuration as a JSON-compatible dict.

        ``defaults`` lists every outstation setting. Each outstation lists its
        name, its host and only the settings that differ from the defaults.
        """
        shared = self.defaults.describe()
        del shared["name"], shared["host"]
        outstations = []
        for outstation in self.outstations:
            own = outstation.describe()
            entry: dict[str, Any] = {"name": own.pop("name"), "host": own.pop("host")}
            for key, value in own.items():
                if value == shared[key]:
                    continue
                if isinstance(value, dict) and isinstance(shared[key], dict):
                    value = {k: v for k, v in value.items() if v != shared[key][k]}
                entry[key] = value
            outstations.append(entry)
        return {
            "allow_control": self.allow_control,
            "bind": self.bind,
            "tables": self.tables,
            "connect_wait": self.connect_wait,
            "capture": self.capture,
            "capture_max_mb": self.capture_max_mb,
            "capture_keep": self.capture_keep,
            "log_file": self.log_file,
            "log_max_mb": self.log_max_mb,
            "log_keep": self.log_keep,
            "log_level": self.log_level,
            "evaluate": self.evaluate.describe(),
            "defaults": shared,
            "outstations": outstations,
        }

    def render(self) -> str:
        """Return the complete configuration as formatted JSON."""
        return settings.render(self.describe())

from_mapping `classmethod` ¶

from_mapping(given: Mapping[str, Any]) -> MasterConfig

Build a configuration from a decoded JSON document.

Raises :class:ConfigError for an unknown setting, a value of the wrong type, an outstation with no name or host, or a name used twice.

Source code in src/py1815/master/config.py

@classmethod
def from_mapping(cls, given: Mapping[str, Any]) -> MasterConfig:
    """Build a configuration from a decoded JSON document.

    Raises :class:`ConfigError` for an unknown setting, a value of the
    wrong type, an outstation with no name or host, or a name used twice.
    """
    settings.section(given, "configuration", _TOP_LEVEL)
    values: dict[str, Any] = {}
    if "allow_control" in given:
        values["allow_control"] = settings.boolean(given["allow_control"], "allow_control")
    for key in ("bind", "tables", "capture", "log_file"):
        if given.get(key) is not None:
            values[key] = settings.text(given[key], key)
    for key in ("capture_max_mb", "log_max_mb"):
        if key in given:
            values[key] = settings.seconds(given[key], key, unit="megabytes")
    if "capture_keep" in given:
        values["capture_keep"] = settings.integer(
            given["capture_keep"], "capture_keep", 0, 1000
        )
    if "log_keep" in given:
        # At least 1: Python's rotating log handler does not rotate with 0,
        # and the file would then grow without limit.
        values["log_keep"] = settings.integer(given["log_keep"], "log_keep", 1, 1000)
    if "log_level" in given:
        level = given["log_level"]
        if level not in LOG_LEVELS:
            raise ConfigError(f"log_level must be one of {', '.join(LOG_LEVELS)}")
        values["log_level"] = level
    if "connect_wait" in given:
        wait = given["connect_wait"]
        if (
            isinstance(wait, bool)
            or not isinstance(wait, (int, float))
            or not math.isfinite(wait)
            or wait < 0
        ):
            raise ConfigError("connect_wait must be a number of seconds, zero or more")
        values["connect_wait"] = float(wait)
    if "evaluate" in given:
        values["evaluate"] = EvaluateConfig.from_mapping(given["evaluate"])

    shared = given.get("defaults", {})
    for required in ("name", "host"):
        if isinstance(shared, Mapping) and required in shared:
            raise ConfigError(f"defaults.{required} belongs in an outstation's own entry")
    defaults = OutstationConfig().changed(shared, "defaults")
    values["defaults"] = defaults

    entries = given.get("outstations", [])
    if isinstance(entries, (str, bytes, Mapping)) or not isinstance(entries, Sequence):
        raise ConfigError("outstations must be a list")
    outstations: list[OutstationConfig] = []
    for position, entry in enumerate(entries):
        where = f"outstations[{position}]"
        outstation = defaults.changed(entry, where)
        for required in ("name", "host"):
            if not getattr(outstation, required):
                raise ConfigError(f"{where}.{required} is required")
        if any(outstation.name == other.name for other in outstations):
            raise ConfigError(f"{where}.name {outstation.name!r} is used more than once")
        outstations.append(outstation)
    values["outstations"] = tuple(outstations)
    return cls(**values)

load `classmethod` ¶

load(path: str | Path) -> MasterConfig

Read a configuration from a JSON file.

Source code in src/py1815/master/config.py

@classmethod
def load(cls, path: str | pathlib.Path) -> MasterConfig:
    """Read a configuration from a JSON file."""
    return cls.from_mapping(read(path))

describe ¶

describe() -> dict[str, Any]

Return the complete configuration as a JSON-compatible dict.

defaults lists every outstation setting. Each outstation lists its name, its host and only the settings that differ from the defaults.

Source code in src/py1815/master/config.py

def describe(self) -> dict[str, Any]:
    """Return the complete configuration as a JSON-compatible dict.

    ``defaults`` lists every outstation setting. Each outstation lists its
    name, its host and only the settings that differ from the defaults.
    """
    shared = self.defaults.describe()
    del shared["name"], shared["host"]
    outstations = []
    for outstation in self.outstations:
        own = outstation.describe()
        entry: dict[str, Any] = {"name": own.pop("name"), "host": own.pop("host")}
        for key, value in own.items():
            if value == shared[key]:
                continue
            if isinstance(value, dict) and isinstance(shared[key], dict):
                value = {k: v for k, v in value.items() if v != shared[key][k]}
            entry[key] = value
        outstations.append(entry)
    return {
        "allow_control": self.allow_control,
        "bind": self.bind,
        "tables": self.tables,
        "connect_wait": self.connect_wait,
        "capture": self.capture,
        "capture_max_mb": self.capture_max_mb,
        "capture_keep": self.capture_keep,
        "log_file": self.log_file,
        "log_max_mb": self.log_max_mb,
        "log_keep": self.log_keep,
        "log_level": self.log_level,
        "evaluate": self.evaluate.describe(),
        "defaults": shared,
        "outstations": outstations,
    }

render ¶

render() -> str

Return the complete configuration as formatted JSON.

Source code in src/py1815/master/config.py

def render(self) -> str:
    """Return the complete configuration as formatted JSON."""
    return settings.render(self.describe())

read ¶

read(path: str | Path) -> dict[str, Any]

Read a JSON configuration file and return the decoded document.

Source code in src/py1815/settings.py

def read(path: str | pathlib.Path) -> dict[str, Any]:
    """Read a JSON configuration file and return the decoded document."""
    try:
        content = pathlib.Path(path).read_text(encoding="utf-8")
    except OSError as error:
        raise ConfigError(f"{path}: {error.strerror or error}") from None
    try:
        document = json.loads(content)
    except ValueError as error:
        raise ConfigError(f"{path} is not valid JSON: {error}") from None
    if not isinstance(document, dict):
        raise ConfigError(f"{path} must hold a JSON object")
    return document

The description of the API¶

py1815.master.openapi ¶

The master's HTTP API, described as an OpenAPI document.

One description of every operation: what it takes, what it returns, and an example of asking for it. :func:build turns that into an OpenAPI 3.1 document with a route for each operation, and the document is committed beside the console, which serves it at /openapi.json.

The description is checked against the service and not trusted: a test fails when an operation exists that is not described here, or is described here and does not exist, and when the committed document is not the one this module builds.

python -m py1815.master.openapi            # print the document
python -m py1815.master.openapi --write    # write it where it is served from
python -m py1815.master.openapi --check    # fail if the committed one is stale

build ¶

build() -> dict[str, Any]

The OpenAPI document, as a dictionary.

Source code in src/py1815/master/openapi.py

def build() -> dict[str, Any]:
    """The OpenAPI document, as a dictionary."""
    paths: dict[str, Any] = {}
    for name, operation in OPERATIONS.items():
        described: dict[str, Any] = {
            "operationId": name,
            "summary": operation["summary"],
            "tags": [operation["tag"]],
            "requestBody": {
                "required": bool(operation["params"].get("required")),
                "content": {
                    "application/json": {
                        "schema": operation["params"],
                        "example": operation["example"],
                    }
                },
            },
            "responses": _responses(operation["result"]),
        }
        if "description" in operation:
            described["description"] = operation["description"]
        paths[f"/api/{name}"] = {"post": described}

    paths["/api"] = {
        "post": {
            "operationId": "message",
            "summary": "Any operation, as one message",
            "description": "The form the line service uses: the operation by name, its "
            "parameters, and an id the answer repeats. Every operation above can be asked "
            "this way.",
            "tags": ["Service"],
            "requestBody": {
                "required": True,
                "content": {
                    "application/json": {
                        "schema": _object(
                            {
                                "id": {"description": "Repeated in the answer."},
                                "op": {"type": "string", "enum": list(OPERATIONS)},
                                "outstation": _OUTSTATION_NAME,
                                "params": {"type": "object"},
                            },
                            required=["op"],
                        ),
                        "example": {
                            "id": 7,
                            "op": "scan",
                            "outstation": "lab",
                            "params": {"kind": "integrity"},
                        },
                    }
                },
            },
            "responses": _responses({"description": "The result of the operation named."}),
        }
    }
    paths["/events"] = {
        "get": {
            "operationId": "events_stream",
            "summary": "What happens unasked, as server-sent events",
            "description": "A stream that stays open. Each event's data is a JSON object "
            "with an `event` field: `frame`, `exchange`, `unsolicited`, `connection` or "
            "`outstations`, and the `outstation` it is about. A subscriber that falls 10,000 "
            "updates behind has its waiting updates replaced by one `lost` event whose "
            "`dropped` field says how many were dropped; read the state again, for example "
            "with `status`, `values`, `events` and `trace`. A stream that has not accepted "
            "a write for 60 seconds is closed.",
            "tags": ["Service"],
            "responses": {
                "200": {
                    "description": "The stream.",
                    "content": {"text/event-stream": {"schema": _STRING}},
                },
                **{
                    status: {"description": description}
                    for status, description in _REFUSALS.items()
                    if status != "400"
                },
            },
        }
    }
    return {
        "openapi": "3.1.0",
        "info": {
            "title": "py1815 master",
            "version": API_VERSION,
            "summary": "A DNP3 master for exercising outstations, as a service.",
            "description": "Every operation of the master, each as a route that takes its "
            "parameters as a JSON object and answers with a JSON object.\n\n"
            "An outstation that does not answer is not an error: the exchange's `outcome` "
            "is `timeout`. An answer has `ok: false` only when the message could not be "
            "acted on, or there was no connection to ask over.\n\n"
            "The service listens on the machine it runs on unless it was given a token, "
            "which every request then carries as `Authorization: Bearer`, or as a `token` "
            "query parameter where a header cannot be set.",
            "license": {"name": "Apache-2.0", "identifier": "Apache-2.0"},
        },
        "servers": [{"url": "http://127.0.0.1:8815", "description": "The default."}],
        "tags": [{"name": name, "description": description} for name, description in _TAGS],
        "paths": paths,
        "components": {
            "schemas": SCHEMAS,
            "securitySchemes": {
                "token": {
                    "type": "http",
                    "scheme": "bearer",
                    "description": "Required only when the service was started with a token.",
                }
            },
        },
        "security": [{}, {"token": []}],
    }

render ¶

render() -> str

The document as it is committed: stable, and readable in a diff.

Source code in src/py1815/master/openapi.py

def render() -> str:
    """The document as it is committed: stable, and readable in a diff."""
    return json.dumps(build(), indent=2, sort_keys=False) + "\n"

Requests¶

py1815.master.requests ¶

The requests a master reads with, as the object headers that ask for them.

Kept apart from anything that sends, so the same request is built whether it goes over a socket or straight into a session, and so each can be pinned to its octets in a test.

write_time ¶

write_time(milliseconds: int) -> bytes

The object of a time write: milliseconds since the epoch, UTC, in 48 bits.

Source code in src/py1815/master/requests.py

def write_time(milliseconds: int) -> bytes:
    """The object of a time write: milliseconds since the epoch, UTC, in 48 bits."""
    return _time_object(ABSOLUTE_TIME, milliseconds)

write_recorded_time ¶

write_recorded_time(milliseconds: int) -> bytes

The object of the write that ends the LAN procedure (IEEE 1815-2012 10.3.3.2).

Group 50 variation 3: the time, in milliseconds since the epoch, at which the master sent the request to record the current time.

Source code in src/py1815/master/requests.py

def write_recorded_time(milliseconds: int) -> bytes:
    """The object of the write that ends the LAN procedure (IEEE 1815-2012 10.3.3.2).

    Group 50 variation 3: the time, in milliseconds since the epoch, at which
    the master sent the request to record the current time.
    """
    return _time_object(LAST_RECORDED_TIME, milliseconds)

clear_restart ¶

clear_restart() -> bytes

The object that clears the restart indication: bit 7 of the indications, written as 0.

Source code in src/py1815/master/requests.py

def clear_restart() -> bytes:
    """The object that clears the restart indication: bit 7 of the indications, written as 0."""
    return bytes(
        [80, 1, QualifierCode.UINT8_START_STOP, DEVICE_RESTART_INDEX, DEVICE_RESTART_INDEX, 0]
    )

freeze_counters ¶

freeze_counters() -> bytes

The header of a freeze: every counter.

Source code in src/py1815/master/requests.py

def freeze_counters() -> bytes:
    """The header of a freeze: every counter."""
    return all_objects_header(STATIC_GROUPS[PointType.COUNTER], DEFAULT_VARIATION)

class_scan ¶

class_scan(*classes: int) -> bytes

The headers of a read of the named classes, in the order named.

Source code in src/py1815/master/requests.py

def class_scan(*classes: int) -> bytes:
    """The headers of a read of the named classes, in the order named."""
    if not classes:
        raise ValueError("a scan names at least one class")
    return b"".join(class_header(number) for number in classes)

scan ¶

scan(kind: str) -> bytes

The headers of a scan by name: integrity, events, class0 to class3, or outputs.

Source code in src/py1815/master/requests.py

def scan(kind: str) -> bytes:
    """The headers of a scan by name: integrity, events, class0 to class3, or outputs."""
    if kind == OUTPUTS:
        return read_points({PointType.BINARY_OUTPUT: None, PointType.ANALOG_OUTPUT: None})
    try:
        return class_scan(*SCANS[kind])
    except KeyError:
        raise ValueError(f"{kind!r} is not a scan; one of {', '.join(SCAN_KINDS)}") from None

read_points ¶

read_points(
    points: Mapping[PointType, Sequence[int] | None],
) -> bytes

Build the headers of a read of named points, in the default variations.

Each point type maps to the indices wanted, or to None for every point of that type. Types are asked in the order given.

Indices are sent as start-stop ranges (qualifiers 0x00 and 0x01), one header for each run of consecutive indices, in ascending order. A DNP3 Subset Level 2 outstation is only required to accept ranges in a read. The index-list qualifiers (0x17 and 0x28) are optional at that level, and opendnp3 rejects them.

Source code in src/py1815/master/requests.py

def read_points(points: Mapping[PointType, Sequence[int] | None]) -> bytes:
    """Build the headers of a read of named points, in the default variations.

    Each point type maps to the indices wanted, or to None for every point of
    that type. Types are asked in the order given.

    Indices are sent as start-stop ranges (qualifiers 0x00 and 0x01), one
    header for each run of consecutive indices, in ascending order. A DNP3
    Subset Level 2 outstation is only required to accept ranges in a read. The
    index-list qualifiers (0x17 and 0x28) are optional at that level, and
    opendnp3 rejects them.
    """
    if not points:
        raise ValueError("a read names at least one point type")
    headers = bytearray()
    for point, indices in points.items():
        group = STATIC_GROUPS[point]
        if indices is None:
            headers += all_objects_header(group, DEFAULT_VARIATION)
            continue
        for start, stop in _runs(indices):
            headers += object_header(group, DEFAULT_VARIATION, start=start, stop=stop)
    return bytes(headers)

Reading objects¶

py1815.decode ¶

Reading the objects a response carries.

The rest of this library writes objects; this reads them, which a master has to do and an outstation never does. It is written from the object layouts in the standard and not by turning the encoders in :mod:py1815.objects around. An encoder and a decoder derived from one another agree with each other whatever they get wrong, and the point of reading an outstation is to learn what it sent.

It reads as far as it can and says where it stopped. An object's width is the only way to find the object after it, so a group or variation that is not in the table ends the walk. That is reported, with the octets that were not read, and is not raised: a response that is half understood is still half of what the outstation said.

PointType ¶

Bases: Enum

The kinds of point a master keeps values for.

Source code in src/py1815/decode.py

class PointType(Enum):
    """The kinds of point a master keeps values for."""

    BINARY_INPUT = "bi"
    BINARY_OUTPUT = "bo"
    COUNTER = "counter"
    FROZEN_COUNTER = "frozen"
    ANALOG_INPUT = "ai"
    ANALOG_OUTPUT = "ao"

Layout `dataclass` ¶

How one object of a group and variation is laid out.

Source code in src/py1815/decode.py

@dataclass(frozen=True)
class Layout:
    """How one object of a group and variation is laid out."""

    #: The point type the value belongs to, or None for an object that is not
    #: a point's value: a time, a command.
    point: PointType | None
    #: Whether the object reports a change and not the present value.
    event: bool = False
    #: Whether the object begins with a flag octet.
    flags: bool = True
    #: The ``struct`` format of the number after the flags, or None for a
    #: binary object, whose state is a bit of the flag octet.
    number: str | None = None
    time: _Time = _Time.NONE
    #: One bit per object and no flags: the packed binary variations.
    packed: bool = False
    #: For an object this module walks past and does not interpret.
    opaque: int = 0

    @property
    def size(self) -> int:
        """The octets one object occupies. Meaningless for a packed layout."""
        if self.opaque:
            return self.opaque
        size = 1 if self.flags else 0
        if self.number is not None:
            size += struct.calcsize(self.number)
        if self.time is _Time.ABSOLUTE:
            size += TIME_SIZE
        elif self.time is _Time.RELATIVE:
            size += 2
        return size

size `property` ¶

size: int

The octets one object occupies. Meaningless for a packed layout.

DecodedObject `dataclass` ¶

One object out of a response.

Source code in src/py1815/decode.py

@dataclass(frozen=True)
class DecodedObject:
    """One object out of a response."""

    group: int
    variation: int
    #: None for an object sent under a count with no index: a time.
    index: int | None
    #: A state, a count, an analog value, a number of milliseconds, or None
    #: for an object this module walks past.
    value: bool | int | float | None
    #: The flag octet, without the state bit of a binary object. None for a
    #: variation that carries no flags, which is not the same as none set.
    flags: int | None
    #: Milliseconds since the epoch, for an object that carries a time or
    #: counts from a common time of occurrence. None otherwise.
    time_ms: int | None
    #: Whether the outstation's clock had been set when it stamped a
    #: relative-time event. None where the object does not say.
    synchronized: bool | None
    point: PointType | None
    event: bool
    #: The object's own octets, without header, index or prefix.
    raw: bytes

Decoded `dataclass` ¶

What was read from a response body, and what was not.

Source code in src/py1815/decode.py

@dataclass(frozen=True)
class Decoded:
    """What was read from a response body, and what was not."""

    objects: tuple[DecodedObject, ...]
    #: The octets from the first thing that could not be read to the end.
    unread: bytes = b""
    #: Why reading stopped there, or None if the whole body was read.
    problem: str | None = None

    @property
    def complete(self) -> bool:
        return self.problem is None

decode_objects ¶

decode_objects(body: bytes) -> Decoded

Every object in a response body that can be read, in the order sent.

Reading stops at the first header or object that cannot be read, and the result says why and holds the octets from there on. Nothing read before that point is discarded: each object was complete, and the outstation sent it.

A common time of occurrence applies to the relative-time events after it in the same body, as the standard has it, and to none before it.

Source code in src/py1815/decode.py

def decode_objects(body: bytes) -> Decoded:
    """Every object in a response body that can be read, in the order sent.

    Reading stops at the first header or object that cannot be read, and the
    result says why and holds the octets from there on. Nothing read before
    that point is discarded: each object was complete, and the outstation
    sent it.

    A common time of occurrence applies to the relative-time events after it
    in the same body, as the standard has it, and to none before it.
    """
    objects: list[DecodedObject] = []
    common = _CommonTime()
    offset = 0
    while offset < len(body):
        try:
            block, end = _block(body, offset, common)
        except _Stop as stop:
            return Decoded(tuple(objects), bytes(body[offset:]), str(stop))
        objects.extend(block)
        offset = end
    return Decoded(tuple(objects))

Master¶

Over a socket¶

py1815.master.api ¶

RetryReport ¶

failed ¶

NotConnected ¶

Outstation ¶

tasks property writable ¶

tasks_due property ¶

unsolicited property ¶

scan_intervals property ¶

connect async ¶

idle async ¶

close async ¶

wait_for async ¶

repeat_scan ¶

Master ¶

outstations property ¶

add async ¶

remove async ¶

close async ¶

The operations¶

py1815.master.operations ¶

Steps ¶

next ¶

result ¶

Operations ¶

operate ¶

write_time ¶

synchronize_time ¶

broadcast ¶

clear_restart ¶

freeze ¶

restart ¶

request ¶

scan ¶

integrity_poll ¶

enable_unsolicited ¶

disable_unsolicited ¶

read ¶

broadcast_address ¶

Commands¶

py1815.master.controls ¶

Mode ¶

Command dataclass ¶

encoded property ¶

PointStatus dataclass ¶

Operated dataclass ¶

operated property ¶

accepted property ¶

status ¶

Plan ¶

next ¶

result ¶

binary ¶

analog ¶

commands ¶

encode ¶

read_statuses ¶

An IEEE 1815.2 DER¶

py1815.master.profile ¶

Function dataclass ¶

points property ¶

curve_settings property ¶

Reading dataclass ¶

value property ¶

state property ¶

quality property ¶

Readings dataclass ¶

Setpoint dataclass ¶

sent_value property ¶

matches property ¶

Written dataclass ¶

accepted property ¶

verified property ¶

Switched dataclass ¶

enabled property ¶

Curve dataclass ¶

CurveWritten dataclass ¶

stopped_at property ¶

tasks `property` `writable` ¶

tasks_due `property` ¶

unsolicited `property` ¶

scan_intervals `property` ¶

connect `async` ¶

idle `async` ¶

close `async` ¶

wait_for `async` ¶

outstations `property` ¶

add `async` ¶

remove `async` ¶

close `async` ¶

Command `dataclass` ¶

encoded `property` ¶

PointStatus `dataclass` ¶

Operated `dataclass` ¶

operated `property` ¶

accepted `property` ¶

Function `dataclass` ¶

points `property` ¶

curve_settings `property` ¶

Reading `dataclass` ¶

value `property` ¶

state `property` ¶

quality `property` ¶

Readings `dataclass` ¶

Setpoint `dataclass` ¶

sent_value `property` ¶

matches `property` ¶

Written `dataclass` ¶

accepted `property` ¶

verified `property` ¶

Switched `dataclass` ¶

enabled `property` ¶

Curve `dataclass` ¶

CurveWritten `dataclass` ¶

stopped_at `property` ¶

accepted `property` ¶

matches `property` ¶

FunctionState `dataclass` ¶

Functions `dataclass` ¶

Declared `dataclass` ¶

Comparison `dataclass` ¶

limits `staticmethod` ¶

span `staticmethod` ¶

Survey `dataclass` ¶

run_async `async` ¶

Report `dataclass` ¶

failed `property` ¶

evaluate `async` ¶

Pause `dataclass` ¶

Check `dataclass` ¶

Result `dataclass` ¶

require `staticmethod` ¶