Metadata-Version: 2.1
Name: mmprovision
Version: 0.7.0+halowlink.1.0.3
Summary: Morse Micro chip provisioning over JTAG / USB / SPI.
License: Proprietary
Author: Sahand Maleki
Author-email: sahand.maleki@morsemicro.com
Requires-Python: >=3.9,<4.0
Classifier: License :: Other/Proprietary License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: pyelftools (>=0.31,<0.32)
Requires-Dist: pyserial (>=3.5,<4.0)
Requires-Dist: pyusb (>=1.2,<2.0)
Requires-Dist: pyyaml (>=6.0,<7.0)
Requires-Dist: xmodem (>=0.4.7,<0.5.0)
Description-Content-Type: text/markdown

# mmprovision

A minimal, self-contained Python library and CLI for provisioning Morse Micro
chips. It boots an on-chip provisioning firmware, then talks YAPS over
JTAG / USB / SPI to flash images, create factory configuration, and write OTP.

## Requirements

- Python ≥ 3.9
- [Poetry](https://python-poetry.org/) for dependency management
- `openocd` on `$PATH` or `$OPENOCD_PATH` (JTAG transport only)
- `openssl` on `$PATH` (`provision_factory_partition` only)

## Install

```bash
cd tools/mmprovision
poetry install
```

Poetry creates a virtualenv and installs an `mmprovision` CLI entry point.
Subsequent commands run inside the env:

```bash
poetry run mmprovision --help
# or drop into a shell with the env active:
poetry shell
```

Source-tree and editable installs follow `_configstore.py` into the adjacent
`mhs` submodule. Release wheels contain a frozen copy instead.

## Build a release wheel

Use the release builder rather than calling `poetry build` directly. It creates
an isolated staging tree, freezes the selected configstore implementation into
the package, and builds the wheel without modifying the source checkout:

```bash
poetry run python build_wheel.py
```

`--configstore` defaults to the `_configstore.py` symlink. Pass it explicitly
to build from a different configstore implementation.

Use `--version` to append a local version extension for one build without
changing the base version in the checkout. For example:

```bash
poetry run python build_wheel.py --version customer1
```

With a base version of `0.2.0`, this produces `0.2.0+customer1`.

To also create a release ZIP containing the wheel, flash image, and recipe:

```bash
poetry run python build_wheel.py \
    --flash-image mm8108_spi_flash_image.bin \
    --recipe recipe.rd17v1.yaml \
    --recipe recipe.rd17v2.yaml
```

The two options must be supplied together. Recipe filenames are resolved from
the `recipes/` directory, and `--recipe` may be repeated or followed by
multiple filenames. The ZIP is written beside the wheel in `dist/`, with the
wheel, flash image, all selected recipes, and a short setup `README.md` at the
archive root.

A typical release directory contains the wheel plus the recipe-specific files:

```text
release/
├── mmprovision-VERSION-py3-none-any.whl
├── mmprovision-VERSION-py3-none-any.zip
├── recipe.yaml
└── application-firmware.bin
```

The chip-boot provisioning images and frozen configstore are inside the wheel;
the application image and recipe remain external so they can vary per release.

## CLI

The tool is driven entirely by a YAML **recipe**. The recipe selects the
transport, picks the firmware to boot, and lists the chip operations. The tool
boots the on-chip provisioning firmware first, then runs the recipe's
operations top-to-bottom, in the order written.

```bash
poetry run mmprovision recipes/RECIPE.yaml
```

The single positional argument is the path to the YAML recipe. `--version`
prints the tool version and exits; `--help` lists the available options.

Logging verbosity is controlled by repeating `-v` (via the standard `logging`
module):

| Flag | Level | Shows |
|---|---|---|
| _(none)_ | `ERROR` | errors only |
| `-v` | `WARNING` | + warnings |
| `-vv` | `INFO` | + per-operation progress |
| `-vvv` | `DEBUG` | + ELF segment addresses, OpenOCD `-d3` / stderr |

```bash
poetry run mmprovision -vv RECIPE.yaml    # show progress
```

Log lines are colour-coded by level when the output is a terminal. Colour is
disabled automatically when stderr is not a TTY (e.g. piped to a file) or when
the [`NO_COLOR`](https://no-color.org) environment variable is set.

### Log file

Every run also writes a plain-text, **full-detail (`DEBUG`) log** — independent
of the console `-v` level — to a timestamped file
(`mmprovision-YYYYMMDD-HHMMSS.log`) inside a log directory:

```bash
poetry run mmprovision --log-dir path/to/logs RECIPE.yaml
```

The directory defaults to `logs/` (relative to the current directory) and is
**created automatically** (including parents) if it does not exist. Pass
`--log-dir ""` to disable the file log entirely.

See [`recipe.example.yaml`](recipes/recipe.example.yaml) for an annotated template.

### DFU

A chip sitting in its DFU program, rather than sitting in the bootloader,
enumerates as a CDC-ACM serial port under Morse's USB vendor id (`0x325B`)
with product id `0x8101`. In DFU the chip can't yet respond to register
reads or process packets, so its chip id can't be auto-detected. Use
`--attempt-dfu-to-provision` to detect and recover from this state before the
recipe's normal boot sequence runs:

```bash
poetry run mmprovision --attempt-dfu-to-provision mm8108 RECIPE.yaml
```

`TARGET` (`mm8108` or `mm8118`) names which bundled SPI flash image to stream
over XMODEM — see [Firmware selection](#firmware-selection) below. If the
device isn't actually in DFU mode, this is a no-op and the recipe proceeds
through its normal boot sequence unchanged.

To use the DFU program of a provisioned chip to update its firmware, use the 
standalone `mmdfu` command instead; see [`mmdfu.md`](mmdfu.md).

## Recipe format

A recipe is a single YAML mapping with three top-level keys:

```yaml
transport:            # how to reach the chip
  type: jtag          # jtag | usb | spi
  # jtag-only, all optional:
  config: ../../targets/eaglecrest/eaglecrestb/jtag_config/openocd_host_only.cfg
  tcl_dir: /path/to/tcl
  openocd: /path/to/openocd
  serial: "FTXXXXXX"

firmware: auto         # "auto" (or omitted) = auto-detect; or a path to an ELF/bin

operations:            # executed in this exact order
  - op: flash
    path: boot.bin
    offset: 0x000000
  - op: flash
    path: payload.bin
    offset: 0x040000
    ask_for_confirmation: true    # prompt Y/N before this step
  - op: otp
    bank: 2
    value: 0x12345678
```

Notes:

- **Order is preserved** — `operations` is a list, so steps run exactly as
  listed (flash and OTP may be interleaved freely).
- **Hex values** are written unquoted with a `0x` prefix (`0x40000`); YAML
  parses these as integers natively, as it does plain decimal integers.
- **`offset`** is optional for `flash` and defaults to `0x0`.
- **`configstore_py`** must be tagged `!path` (e.g. `configstore_py: !path
  custom/configstore.py`) so it is parsed as a filesystem path; relative paths
  are resolved relative to the directory from which `mmprovision` is run.
- **`ask_for_confirmation: true`** on any step prompts for `Y/N` before it runs;
  declining aborts the whole run. Defaults to `false`.

### Operations

| `op` | Keys | Effect |
|---|---|---|
| `flash` | `path`, `offset` (default `0x0`) | Flash a binary file at flash `offset`. |
| `otp` | `bank`, `value` | Write `value` to OTP `bank`. |
| `io_verify` | `gpio`; optional `io_type`, `value`, `wait_delay_in_ms`, `label` | Verify or drive one GPIO. `io_type: !gpio_io_type INPUT` (the default) waits for the pin to reach `value` and then return to the opposite level. `OUTPUT` drives it to `value` for `wait_delay_in_ms` and restores it. `label`, if given, names the pin in the operator prompt (e.g. `"DFU/DPP button"`) instead of the bare GPIO number. |
| `validate_leds` | none; optional `display_time_ms` (default `300`), `value`, `leds` | Light every LED in the recipe's top-level `leds` map, then ask the operator to confirm. See below. |
| `provision_factory_partition` | `keys`; optional `configstore_py`, `offset` | Generate a fresh factory partition using the bundled configstore by default, then flash it at `offset`. An explicit `configstore_py` is a strict override. |
| `security_state_change` | `state`; optional `program_dev_key_hash` (default `false`) | Change the chip security `state`. `state` is a name (`raw`, `development`, `secured`, `production`, `end_of_life`). Set `program_dev_key_hash: true` to also program the development public-key hash. |

Any step may also carry `ask_for_confirmation: true` (see notes above).

#### LED validation: `validate_leds`

Drives each pin in the recipe's top-level `leds` map (a sibling of
`transport`/`operations`, described under
[`keys`](#factory-partition-config-keys) below) as an output for
`display_time_ms`, naming each LED as it lights so the operator can check the
right one came on.

```
LED 1/5: led_status green on GPIO6 (physical GPIO0 pad)
LED 2/5: led_status red on GPIO1
LED 3/5: led_status blue on GPIO2
LED 4/5: led_dfu blue on GPIO3
LED 5/5: led_power green on GPIO4
```

The order is ascending GPIO number, except **GPIO6 first** to match the default
driverless USB pin-mux.

#### Factory partition config: `keys`

`provision_factory_partition` builds the config store from a single `keys` map,
keyed by the exact config-store key:

- A **plain value** is stored verbatim as a config-store string. Values are
  stringified; a YAML bool becomes `"true"`/`"false"` (what `mmconfig_read_bool`
  expects). Use this for settings like `wlan.country_code`, `wlan.boot_mode`,
  and `dlusb_led.on_during_suspend`.
- A **tagged value** is resolved host-side before provisioning. An unknown tag
  is a hard error (PyYAML `ConstructorError`). Recognised tags:
  - `!dpp_bootstrap_key` — generate and store a fresh DPP QR bootstrap key,
    as a hex string (used as a valueless tag; omit the key to leave it
    unprovisioned).
  - `!pin_pattern` — takes a `{pin: ..., pattern: ...}` mapping and produces
    one `"<pin>:<pattern>"` entry, the grammar `dlusb_led.state_*` /
    `dlusb_led.power_normal` / `dlusb_led.power_suspend` use for a single pin.
  - `!join` — takes a YAML sequence and joins it into one comma-separated
    string, e.g. for a `dlusb_led.state_*`/`dlusb_led.power_*` entry driving
    more than one pin (each item a `!pin_pattern`), or for a
    `dfu.led_state_*`/`dfu.led_power` entry driving more than one raw pin.
    Generic — not specific to LEDs.

  `pin` fields (and any raw pin in a `!join` list) are typically written as a
  reference into the recipe's top-level `leds` map (a sibling of
  `transport`/`operations`) rather than a bare GPIO number, using a plain YAML
  anchor/alias. See `dlusb_led_config.py` for the states/limits and
  `../dlusb_led_visualiser/dlusb_led_visualiser.html` for previewing the result.

```yaml
leds:
  led_status:
    red: &led_status_red "1"
    green: &led_status_green "6"

transport:
  type: usb

operations:
  - op: provision_factory_partition
    offset: 0x30000
    keys:
      wlan.country_code: US
      wlan.boot_mode: dpp_qr
      wlan.dpp_qr_bootstrap_key: !dpp_bootstrap_key
      dlusb_led.pattern_connecting: "0:100,255,100,0"
      dlusb_led.state_connecting: !pin_pattern
        pin: *led_status_green
        pattern: connecting
      dfu.led_state_normal: !join [*led_status_green, *led_status_red]
```

Run `mmprovision --dry-run <recipe>` to exercise the whole recipe without
touching hardware: each operation logs what it would send instead of talking to
the device, and the resulting config-store contents (as `strings`/`hex_strings`
JSON, one block per `provision_factory_partition` op) are logged too. Dry-run
raises the console to INFO by default (equivalent to `-vv`), so this output is
visible without extra flags.

### Firmware selection

If `firmware` is `auto` (or omitted), the tool reads the chip id from address
`0x2D20` and picks the matching firmware from `bin/`:

| Chip id | Firmware |
|---|---|
| `0x809` | `bin/mm8108_provisioning_firmware.bin` |
| `0x909` | `bin/mm8118_provisioning_firmware.bin` |

Set `firmware:` to a path to override the auto-selection with your own
ELF/binary.

`--attempt-dfu-to-provision` (see [DFU](#dfu) above) selects differently: it
can't read the chip id, so `TARGET` names it directly, and it streams a full
SPI flash image rather than the RAM-loaded provisioning firmware above:

| Target | Image |
|---|---|
| `mm8108` | `bin/mm8108_provisioning_spi_flash_image.bin` |
| `mm8118` | `bin/mm8118_provisioning_spi_flash_image.bin` |

### Transports

- **jtag** — accepts the optional `config`, `tcl_dir`, `openocd`, and `serial`
  keys shown above. `config` defaults to
  `targets/eaglecrest/eaglecrestb/jtag_config/openocd_host_only.cfg`; `openocd`
  otherwise falls back to `$OPENOCD_PATH` or `openocd` on `$PATH`.
- **usb** — no extra keys; pyusb auto-discovers VID `0x325B`. The CLI fails
  fast with a clear error if more than one device with that vendor id is
  present — matched on VID alone, so a normal-mode device and a device
  sitting in DFU mode plugged in together both count; only one is supported
  at a time.
- **spi** — not supported via the CLI (the bus protocol lives outside this
  package). The recipe exits with an error; use the Python API with
  `SPITransport` instead (see below).

## Programmatic use

The operations are plain functions. Construct a transport, boot the on-chip
provisioning firmware with `boot()`, then call the functions against it. The
top-level namespace exposes `boot`, the operation functions (`flash`,
`provision_factory_partition`, `write_otp`, `io_verify`, `validate_leds`,
`transmit`), `Recipe`, and the transports
(`OpenOCDTransport`, `USBTransport`, `SPITransport`); packets, `YdsYsl`, and
`RawBinary` are in the submodules. DFU recovery (`attempt_dfu`,
`attempt_dfu_to_provision`; see [DFU](#dfu) above) lives in `mmprovision.dfu`
and isn't part of the top-level namespace either — it's independent of
`Transport`/`boot()`, working directly over the DFU CDC-ACM serial port.

```python
import mmprovision as mm

# Pick a transport
t = mm.OpenOCDTransport(config="/path/to/openocd_host_only.cfg")
# or: t = mm.USBTransport()
# or: t = mm.SPITransport(bus=my_morse_spi_instance)

mm.boot(t)              # auto-detects the chip and boots its bundled firmware

mm.flash(t, "payload.bin", 0x0)
mm.write_otp(t, bank=2, value=0x12345678)
```

To generate and flash a factory partition from Python:

```python
mm.provision_factory_partition(
    t,
    keys={
        "wlan.country_code": "US",
        "wlan.boot_mode": "dpp_qr",
        # A value computed host-side (fresh randomness) uses the same marker
        # the !dpp_bootstrap_key YAML tag produces.
        "wlan.dpp_qr_bootstrap_key": mm.DppBootstrapKey(),
        # !pin_pattern/!join have no runtime side effects, so there's no
        # marker for them -- just supply the resolved string directly.
        "dlusb_led.state_connecting": "6:connecting",
    },
    offset=0x30000,
)
```

Source-tree runs follow `_configstore.py` into the adjacent `mhs` submodule,
while release wheels contain the snapshot selected by `build_wheel.py`. Passing
`configstore_py=Path("custom/configstore.py")` explicitly overrides either;
the module must define a `ConfigStorePartition` class with callable
`load_data()` and `serialize()` methods. A missing or incompatible override is
an error and never silently falls back.

`boot(transport)` reads the chip id, picks the matching provisioning image from
this package's `bin/` directory, and boots it — you don't need to know the
firmware filename. (Pass `boot(transport, "my.elf")` to run your own image.)

`flash(transport, path, offset=0)` reads the file, pads it to a 4-byte
boundary, and writes it in `FLASH_CHUNK_SIZE`-byte chunks, one `FlashWrite`
packet each, starting at flash `offset`. `write_otp(transport, bank, value)`
sends a single `OtpWrite`. Both raise if the chip NACKs or times out.

### Sending individual packets

`transmit(transport, packet)` is the primitive both operations build on: it
opens a fresh `YdsYsl` channel, drains any stale RX packets, sends the packet,
waits up to `timeout_s` (default 5s) for the ACK, and validates it — raising on
timeout, a non-zero status, or a packet-id/op-code mismatch. `write_otp` is
just:

```python
from mmprovision import transmit
from mmprovision.packets import OtpWrite

def write_otp(transport, bank, value):
    transmit(transport, OtpWrite(bank, value))
```

### Running a recipe in code

`Recipe` is the optional YAML layer over those functions:

```python
from mmprovision import Recipe, USBTransport, boot

recipe = Recipe.from_yaml("recipe.yaml")
t = USBTransport()
boot(t, recipe.firmware)   # honours the recipe's firmware setting (auto by default)
recipe.run(t)   # calls the operation functions in order, one per recipe step
```

## Wire protocol notes

- YAPS delimiter is a single 32-bit word: `[13:0]` size (raw + 256 reserved
  page offset), `[16:14]` pool ID, `[18:17]` padding, `[19]` IRQ, `[24:20]`
  reserved, `[31:25]` CRC-7.
- Outbound packet header is `<II>` little-endian `(op_code, packet_id)` followed
  by the per-op payload (`FlashWrite`: address, length, content bytes;
  `OtpWrite`: bank, value). Op codes: `FLASH_WRITE = 0`, `OTP_WRITE = 1`.
- Inbound ACK is three 32-bit words: `(op_code, packet_id, status)`. `status`
  is a `morse_error_t` value cast to `uint32_t` on the wire: `0` is success; a
  non-zero value is a negative error code (e.g. `EIO`, `ENOMEM`) that arrives
  wrapped around as a large unsigned int. `AckPacket.status_name()` (used by
  its `__str__`) resolves it back to an errno name when possible, falling
  back to the raw wire value otherwise.
- YDS write address: `0x170000`. YSL read address: `0x170000`. Status
  registers at `0x00178000` (TX pages holding), `0x00178020` (TX queue depth),
  `0x00178038` (RX bytes in queue).
- Firmware boot trigger (all transports): write `1` to `0x00004100`.
- Chip id is read from `0x2D20`.
- SPI digital reset: write `0xDEAD` to `0x000020AC`.

## Extending

### Add a new transport

Subclass `Transport` (or `BulkTransport` if you load via memory writes and
trigger via a register). Extension base classes live in the submodules, not the
top-level namespace:

```python
from mmprovision.transport import BulkTransport

class MyTransport(BulkTransport):
    def write_bulk_words(self, address, words):
        ...
    def read_bulk_words(self, address, count) -> list[int]:
        ...
    def digital_reset(self) -> None:
        ...
```

`load` and `start` come from `BulkTransport`. Override `TRIGGER_REGISTER` to
change the trigger address. `mmprovision.boot()` then works against your
transport unchanged.

### Add a new packet type

```python
import struct
from mmprovision.packets import Packet

class Erase(Packet):
    OP_CODE = 7
    def __init__(self, address, length):
        super().__init__()
        self.address = address
        self.length = length
    def payload(self) -> list[int]:
        return list(struct.pack("<II", self.address, self.length))
```

`Packet.to_byte_list()` already prepends the `(op_code, packet_id)` header.
Send it with `transmit(transport, Erase(0x10000, 0x1000))`.

### Add a new operation

An operation is just a function that takes a `transport` and does the work,
usually via one or more `transmit` calls. It should also accept a keyword-only
`on_progress` callback (defaulting to a no-op) and call it with `(done, total)`
as it advances — a single-shot op reports once on completion:

```python
from mmprovision import transmit

def _noop(done, total): ...

def erase(transport, address, length, *, on_progress=_noop):
    transmit(transport, Erase(address, length))
    on_progress(1, 1)
```

That is immediately usable from Python. To also expose it in YAML recipes, add
an entry to the `_OPS` table in `recipe.py` — mapping the recipe `op` name to
the function.

```python
# recipe.py
_OPS = {
    "flash": flash,
    "otp": write_otp,
    "erase": erase,
}
```

It is then available as an operation in any recipe (argument names match the
function's parameters):

```yaml
operations:
  - op: erase
    address: 0x10000
    length: 0x1000
```

## Building the provisioning firmware

The images in `bin/` are built from the `mmprovision` host program against the
corewait MAC/PHY programs; copy the resulting build artefact into `bin/`.

For the MM8118 firmware (`ROM=b3`):

```bash
make -C targets/eaglecrest/eaglecrestb -j$(nproc) clean HOST_PROGRAM=mmprovision MAC_PROGRAM=corewait UPHY_PROGRAM=corewait LPHY_PROGRAM=corewait all ROM=b3 EXCLUDE_SECURE_BOOTLOADER_FROM_COMBINED_BIN=1
cp targets/eaglecrest/eaglecrestb/build/mm8118b3-rl.bin tools/mmprovision/bin/mm8118_provisioning_firmware.bin
cp targets/eaglecrest/eaglecrestb/build/mm8118_spi_flash_image.bin tools/mmprovision/bin/mm8118_provisioning_spi_flash_image.bin
```

For the MM8108 firmware (`ROM=b2`):

```bash
make -C targets/eaglecrest/eaglecrestb -j$(nproc) clean HOST_PROGRAM=mmprovision MAC_PROGRAM=corewait UPHY_PROGRAM=corewait LPHY_PROGRAM=corewait all ROM=b2 EXCLUDE_SECURE_BOOTLOADER_FROM_COMBINED_BIN=1
cp targets/eaglecrest/eaglecrestb/build/mm8108b2-rl.bin tools/mmprovision/bin/mm8108_provisioning_firmware.bin
cp targets/eaglecrest/eaglecrestb/build/mm8108_spi_flash_image.bin tools/mmprovision/bin/mm8108_provisioning_spi_flash_image.bin
```

