NDTS Gateway Installation

Gateway Installation

Prerequisites

• A Gateway UUID supplied by the NAN Repository administrator
• A Linux host (Ubuntu 20.04 or later recommended) reachable by all workstations
• TCP port 60195 open inbound from those workstations

Installation Steps

# Copy the .gz file to the gateway computer
tar xf data-transport-gateway-<version>.tar.gz
cd dtgateway-installer
sudo ./install.sh
# → You will be prompted only for the Gateway UUID

Firewall Note

Open port 60195 to each NDTS workstation; no other inbound ports are required.

Verifying the Service

sudo systemctl status data-transport-gateway      # should show “active (running)”
sudo systemctl start  data-transport-gateway      # start if not running

Updating the Gateway

# Run the same installer script from a newer package
cd dtgateway-installer
sudo ./install.sh             # auto-detects an existing install; no prompts

Follow the directions above for verifying the service after updating the Gateway to ensure that the service is running.

Uninstalling the Gateway

sudo ./uninstall.sh           # script is included inside every package

Rolling Back the Gateway

Obtain the desired older package and run its install.sh. Rolling back more than one or two versions is not recommended.

NDTS Gateway Configuration

The gateway reads its settings from /opt/nandt-gateway/gateway_configuration.json. The file **must** be valid JSON. Optional keys and their defaults are: :contentReference[oaicite:0]{index=0}

• "log_file_path" – path for the log file (default /opt/nandt-gateway/nan-gateway.log)
• "logging_level" – one of fatal (=critical), error, warning, info, debug (default "info")
• "logging_format" – Python logging format string (default "%(asctime)s %(levelname)s %(module)s [%(process)d]: %(message)s")
• "base_output_path" – where received data are written (default /opt/nandt-gateway/data/received)
• "https_endpoint" – HTTPS endpoint of the NDTS Receiver (default "https://receiver.usnan.org")
• "retry_limit" / "retry_delay" – number of send attempts (default 5) and seconds between retries (default 300)
• "rsync_username" / "rsync_node" – credentials used for the final rsync step (defaults "nandt" and "receiver.usnan.org")
• "request_queue_size" – backlog of outstanding daemon connections (default 50)

{
  "log_file_path": "/opt/nandt-gateway/nan-gateway.log",
  "logging_level": "info",
  "base_output_path": "/opt/nandt-gateway/data/received",
  "https_endpoint": "https://receiver.usnan.org",
  "retry_limit": 5,
  "retry_delay": 300
}

---

NDTS Gateway Components

All files reside under /opt/nandt-gateway unless noted. :contentReference[oaicite:1]{index=1}

• main.py – entry point for the Python service
• /etc/systemd/system/data-transport-gateway.service – systemd unit file
• nan-gateway.log – runtime log (path overridable via log_file_path)
• gateway_configuration.json – configuration described above
• data/
  • received/ – transient storage for incoming experiment data
  • retransmitter.pid – PID of the active retransmitter helper
  • users.dat – current user / sample / project list
  • user_checksum.dat – checksum that governs list refreshes

---

NDTS Operation

• Experiment transfer – Workstation daemons connect on TCP 60195, announce a dataset, and stream it to received/. Successful transfers spawn a **Transmitter** process that forwards the data to the receiver; on success the local copy is deleted. :contentReference[oaicite:2]{index=2}
• Retransmit – At start-up and after each daemon heartbeat, a **Retransmitter** scans received/. If a dataset’s original retry window has expired, it launches a new Transmitter until delivery succeeds or the file is manually removed.
• Heartbeat forwarding – Every daemon heartbeat is wrapped with the gateway UUID, local and UTC timestamps, then sent to the receiver by a short-lived **Heartbeat Transmitter**.
• User list propagation – Every 15 s the gateway requests an updated list from the receiver; if the checksum differs it saves the new list to users.dat and serves it to workstations on demand.

---

NDTS Logging

Logging level is controlled by the "logging_level" key in gateway_configuration.json. Levels (high → low volume): fatal, error, warning, info, debug. Default is info. The log file path defaults to /opt/nandt-gateway/nan-gateway.log. :contentReference[oaicite:3]{index=3}

Example log excerpt:

2022-06-06 09:22:23,130 INFO main [61393]: Starting gateway retransmitter instance...
2022-06-06 09:22:23,132 INFO main [61393]: Starting gateway receiver network endpoint...
2022-06-06 09:23:12,829 INFO GatewayTransmitter [61512]: Heartbeat sent to https://receiver.usnan.org/heartbeat

:contentReference[oaicite:4]{index=4}