NDTS Daemon Operation - Revision history

Mmaciejewski at 20:05, 9 June 2025

2025-06-09T20:05:31Z

← Older revision		Revision as of 20:05, 9 June 2025
Line 1:		Line 1:
			{{NDTS_Navbox}}

	== Overview ==		== Overview ==
	This page explains how to control the '''data-transport-daemon''' service, verify connectivity, and interpret the daemon’s log and audit files on every spectrometer workstation.		This page explains how to control the '''data-transport-daemon''' service, verify connectivity, and interpret the daemon’s log and audit files on every spectrometer workstation.

Mmaciejewski: /* Overview */

2025-06-09T17:35:47Z

Overview

← Older revision		Revision as of 17:35, 9 June 2025
Line 1:		Line 1:
	== Overview ==		== Overview ==
	This page explains how to control the '''data-transport-daemon''' service, verify connectivity, and interpret the daemon’s log and audit files on every spectrometer workstation.		This page explains how to control the '''data-transport-daemon''' service, verify connectivity, and interpret the daemon’s log and audit files on every spectrometer workstation.

			== TopSpin version prior to 4.x ==
			The daemon tracks experiment start/stop times from the TopSpin account file. By default, TopSpin accounting is not enabled. For NAN to properly automatically harvest datasets, TopSpin accounting must be enabled in each workstation users account. Follow these [[Enabling TopSpin Accounting\|instructions]] to enable Accounting.

	== Service Control ==		== Service Control ==

Mmaciejewski: /* Logging */

2025-06-04T13:16:08Z

Logging

@@ Line 67: / Line 67: @@
 == Logging ==
-Logging of the Daemon is performed in different files. (A) The file running_workstation_version-X.Y.Z is used to log the current workstation version. (B) The ndtd_audit.txt file for monitoring actions performed by the Daemon such as when datasets complete and what the harvesting status for the dataset. (C) the nan-dtdaemon.log is used to log status changes needed by the daemon to properly harvest datasets and associate them to the correct NAN users.
+Logging of the Daemon is performed in different files. (A) The <code>file running_workstation_version-X.Y.Z</code> is used to log the current workstation version. (B) The <code>ndtd_audit.txt</code> file for monitoring actions performed by the Daemon such as when datasets complete and what the harvesting status for the dataset. (C) the <code>nan-dtdaemon.log</code> is used to log status changes needed by the daemon to properly harvest datasets and associate them to the correct NAN users.
 === (A) Version Tracking ===
@@ Line 157: / Line 157: @@
 # '''“Topspin program has been detected and is running.”''' — daemon detected that the Topspin acquisition directory running
 # '''“Setting directory to watch to …”''' — shows the location of the Topspin directory which the daemon will watch for file modifications that indicate the start and end of an acquisition

Mmaciejewski: /* (C) Daemon Logging */

2025-06-04T13:14:35Z

← Older revision		Revision as of 13:14, 4 June 2025
Line 109:		Line 109:
	Detailed logging of the daemon for detecting things such as workstation users changing, TopSpin starting, which users version of TopSpin is controlling the spectrometer, when a different NAN user is selected, which files to monitor for experiment completions, and others. The logs are saved to		Detailed logging of the daemon for detecting things such as workstation users changing, TopSpin starting, which users version of TopSpin is controlling the spectrometer, when a different NAN user is selected, which files to monitor for experiment completions, and others. The logs are saved to

	/opt/nan-dtdaemon/logs/nan-dtdaemon.log		<pre>/opt/nan-dtdaemon/logs/nan-dtdaemon.log</pre>

	==== Log Levels ====		==== Log Levels ====

Mmaciejewski: /* Version Tracking */

2025-06-04T13:14:15Z

Version Tracking

← Older revision		Revision as of 13:14, 4 June 2025
Line 67:		Line 67:

	== Logging ==		== Logging ==
	Logging of the Daemon is performed in ~~two~~ different files. The ndtd_audit.txt file for monitoring actions performed by the Daemon such as when datasets complete and what the harvesting status for the dataset. ~~The~~		Logging of the Daemon is performed in different files. (A) The file running_workstation_version-X.Y.Z is used to log the current workstation version. (B) The ndtd_audit.txt file for monitoring actions performed by the Daemon such as when datasets complete and what the harvesting status for the dataset. (C) the nan-dtdaemon.log is used to log status changes needed by the daemon to properly harvest datasets and associate them to the correct NAN users.

	=== Version Tracking ===		=== (A) Version Tracking ===
	* When the Daemon starts it writes the active version number in two places ...		* When the Daemon starts it writes the active version number in two places ...
	<pre>/opt/nan-dtdaemon/running_workstation_version-X.Y.Z</pre>which records the version number and a timestamp of when the daemon started, and		<pre>/opt/nan-dtdaemon/running_workstation_version-X.Y.Z</pre>which records the version number and a timestamp of when the daemon started, and
Line 75:		Line 75:
	which write an INFO message stating the version number and which is time-stamped		which write an INFO message stating the version number and which is time-stamped

	=== Experiment Transfer Audit File ===		=== (B) Experiment Transfer Audit File ===
	Each time the NDTS daemon processes a completed experiment, it logs a detailed audit entry to the file		Each time the NDTS daemon processes a completed experiment, it logs a detailed audit entry to the file
	<pre>/opt/nan-dtdaemon/logs/ndtd_audit.txt</pre>This audit trail provides traceable records of all harvesting actions performed by the daemon on a given spectrometer workstation		<pre>/opt/nan-dtdaemon/logs/ndtd_audit.txt</pre>This audit trail provides traceable records of all harvesting actions performed by the daemon on a given spectrometer workstation
Line 106:		Line 106:
	\|}		\|}

	=== Daemon Logging ===		=== (C) Daemon Logging ===
	Detailed logging of the daemon for detecting things such as workstation users changing, TopSpin starting, which users version of TopSpin is controlling the spectrometer, when a different NAN user is selected, which files to monitor for experiment completions, and others. The logs are saved to		Detailed logging of the daemon for detecting things such as workstation users changing, TopSpin starting, which users version of TopSpin is controlling the spectrometer, when a different NAN user is selected, which files to monitor for experiment completions, and others. The logs are saved to

Mmaciejewski: /* Slack Notifications */

2025-06-04T13:09:01Z

Slack Notifications

← Older revision		Revision as of 13:09, 4 June 2025
Line 67:		Line 67:

	== Logging ==		== Logging ==
	~~asdf~~		Logging of the Daemon is performed in two different files. The ndtd_audit.txt file for monitoring actions performed by the Daemon such as when datasets complete and what the harvesting status for the dataset. The

	=== Version Tracking ===		=== Version Tracking ===
Line 107:		Line 107:

	=== Daemon Logging ===		=== Daemon Logging ===
	~~The NDTS~~ daemon ~~writes~~ a ~~log file~~ to /opt/nan-dtdaemon/logs/~~ndtd_audit~~.~~txt and can be configured to different levels of verbosity~~		Detailed logging of the daemon for detecting things such as workstation users changing, TopSpin starting, which users version of TopSpin is controlling the spectrometer, when a different NAN user is selected, which files to monitor for experiment completions, and others. The logs are saved to

			/opt/nan-dtdaemon/logs/nan-dtdaemon.log

	==== Log Levels ====		==== Log Levels ====

Mmaciejewski at 12:56, 4 June 2025

2025-06-04T12:56:38Z

← Older revision		Revision as of 12:56, 4 June 2025
Line 2:		Line 2:
	This page explains how to control the '''data-transport-daemon''' service, verify connectivity, and interpret the daemon’s log and audit files on every spectrometer workstation.		This page explains how to control the '''data-transport-daemon''' service, verify connectivity, and interpret the daemon’s log and audit files on every spectrometer workstation.

	== ~~'''~~Service Control~~'''~~ ==		== Service Control ==
	<pre>		<pre>
	# Start the daemon		# Start the daemon
Line 18:		Line 18:
	*Note, the daemon will not start again if another instance is already running		*Note, the daemon will not start again if another instance is already running

	== ~~'''~~Heartbeats and Connectivity~~'''~~ ==		== Heartbeats and Connectivity ==

	On a regular basis (by default, every '''10 minutes'''), each NDTS daemon sends a '''heartbeat''' message to the Gateway. These messages serve as a continuous health check and confirm that the daemon is active and communicating. Each heartbeat contains a set of diagnostic and identity information used for system monitoring and troubleshooting and includes:		On a regular basis (by default, every '''10 minutes'''), each NDTS daemon sends a '''heartbeat''' message to the Gateway. These messages serve as a continuous health check and confirm that the daemon is active and communicating. Each heartbeat contains a set of diagnostic and identity information used for system monitoring and troubleshooting and includes:
Line 66:		Line 66:
	* <code>uchc-ndts-notifications</code>		* <code>uchc-ndts-notifications</code>

	== ~~'''~~Logging~~'''~~ ==		== Logging ==
	asdf		asdf

	=== ~~'''~~Version Tracking~~'''~~ ===		=== Version Tracking ===
	* When the Daemon starts it writes the active version number in two places ...		* When the Daemon starts it writes the active version number in two places ...
	<pre>/opt/nan-dtdaemon/running_workstation_version-X.Y.Z</pre>which records the version number and a timestamp of when the daemon started, and		<pre>/opt/nan-dtdaemon/running_workstation_version-X.Y.Z</pre>which records the version number and a timestamp of when the daemon started, and
Line 75:		Line 75:
	which write an INFO message stating the version number and which is time-stamped		which write an INFO message stating the version number and which is time-stamped

	=== ~~'''~~Experiment Transfer Audit File~~'''~~ ===		=== Experiment Transfer Audit File ===
	Each time the NDTS daemon processes a completed experiment, it logs a detailed audit entry to the file		Each time the NDTS daemon processes a completed experiment, it logs a detailed audit entry to the file
	<pre>/opt/nan-dtdaemon/logs/ndtd_audit.txt</pre>This audit trail provides traceable records of all harvesting actions performed by the daemon on a given spectrometer workstation		<pre>/opt/nan-dtdaemon/logs/ndtd_audit.txt</pre>This audit trail provides traceable records of all harvesting actions performed by the daemon on a given spectrometer workstation
Line 106:		Line 106:
	\|}		\|}

	=== ~~'''~~Daemon Logging~~'''~~ ===		=== Daemon Logging ===
	The NDTS daemon writes a log file to /opt/nan-dtdaemon/logs/ndtd_audit.txt and can be configured to different levels of verbosity		The NDTS daemon writes a log file to /opt/nan-dtdaemon/logs/ndtd_audit.txt and can be configured to different levels of verbosity

	==== ~~'''~~Log Levels~~'''~~ ====		==== Log Levels ====
	Each line begins with a level tag. The level is controlled by the		Each line begins with a level tag. The level is controlled by the
	<code>log_level</code> parameter in the configuration file <code>ndtd_configuration.dat</code>.		<code>log_level</code> parameter in the configuration file <code>ndtd_configuration.dat</code>.
Line 129:		Line 129:
	\|}		\|}

	==== ~~'''~~Log File Example~~'''~~ ====		==== Log File Example ====
	The fragment below is reproduced verbatim from the PDF (pp. 14-15):		The fragment below is reproduced verbatim from the PDF (pp. 14-15):

Line 146:		Line 146:
	</pre>		</pre>

	===== ~~'''~~Analysis of the Example~~'''~~ =====		===== Analysis of the Example =====
	# '''"LOG_START"''' — is the first line indicating the new instance of the daemon and the data and time that it started.		# '''"LOG_START"''' — is the first line indicating the new instance of the daemon and the data and time that it started.
	# '''"Workstation version is 1.0.15”''' — indicates the daemon is running and the version number		# '''"Workstation version is 1.0.15”''' — indicates the daemon is running and the version number
Line 156:		Line 156:
	# '''“Setting directory to watch to …”''' — shows the location of the Topspin directory which the daemon will watch for file modifications that indicate the start and end of an acquisition		# '''“Setting directory to watch to …”''' — shows the location of the Topspin directory which the daemon will watch for file modifications that indicate the start and end of an acquisition

	== ~~'''~~Troubleshooting Checklist~~'''~~ ==		== Troubleshooting Checklist ==
	{\| class="wikitable"		{\| class="wikitable"
	! Symptom		! Symptom
Line 174:		Line 174:
	\|}		\|}

	== ~~'''~~Next Step~~'''~~ ==		== Next Step ==
	Return to [[NDTS Overview\|NDTS Overview]] or continue to [[NDTS_Data_Access\|Accessing Collected Data]].		Return to [[NDTS Overview\|NDTS Overview]] or continue to [[NDTS_Data_Access\|Accessing Collected Data]].

Mmaciejewski: /* Running, Monitoring, and Logging the Daemon */

2025-06-04T12:51:59Z

Running, Monitoring, and Logging the Daemon

← Older revision		Revision as of 12:51, 4 June 2025
Line 1:		Line 1:
	== ~~Running, Monitoring, and Logging the Daemon~~ ==		== Overview ==
	This page explains how to control the '''data-transport-daemon''' service, verify connectivity, and interpret the daemon’s log and audit files on every spectrometer workstation.		This page explains how to control the '''data-transport-daemon''' service, verify connectivity, and interpret the daemon’s log and audit files on every spectrometer workstation.

Mmaciejewski: /* Version Tracking */

2025-06-03T17:01:41Z

Version Tracking

← Older revision		Revision as of 17:01, 3 June 2025
Line 70:		Line 70:

	=== '''Version Tracking''' ===		=== '''Version Tracking''' ===
	* When the Daemon starts it writes the active version number in two ~~palces~~ ...		* When the Daemon starts it writes the active version number in two places ...
	<pre>/opt/nan-dtdaemon/running_workstation_version-X.Y.Z</pre>which records the version number and a timestamp of when the daemon started		<pre>/opt/nan-dtdaemon/running_workstation_version-X.Y.Z</pre>which records the version number and a timestamp of when the daemon started, and
	<pre>/opt/nan-dtdaemon/logs/nan-dtdaemon.log</pre>		<pre>/opt/nan-dtdaemon/logs/nan-dtdaemon.log</pre>
	which write an INFO message stating the version number and which is time-stamped		which write an INFO message stating the version number and which is time-stamped
Line 77:		Line 77:
	=== '''Experiment Transfer Audit File''' ===		=== '''Experiment Transfer Audit File''' ===
	Each time the NDTS daemon processes a completed experiment, it logs a detailed audit entry to the file		Each time the NDTS daemon processes a completed experiment, it logs a detailed audit entry to the file
	<pre>/opt/nan-dtdaemon/logs/ndtd_audit.txt</pre>. This audit trail provides traceable records of all harvesting actions performed by the daemon on a given spectrometer workstation.		<pre>/opt/nan-dtdaemon/logs/ndtd_audit.txt</pre>This audit trail provides traceable records of all harvesting actions performed by the daemon on a given spectrometer workstation

	Each line in the audit file includes the following fields:		Each line in the audit file includes the following fields:
Line 105:		Line 105:
	\| <code>skipped-disabled</code> \|\| Experiment was skipped due to daemon or GUI configuration disabling harvesting		\| <code>skipped-disabled</code> \|\| Experiment was skipped due to daemon or GUI configuration disabling harvesting
	\|}		\|}

	~~This file allows administrators and developers to verify the behavior of the daemon and trace any gaps in experiment delivery.~~

	=== '''Daemon Logging''' ===		=== '''Daemon Logging''' ===
	NDTS writes ~~two workstation logs:~~		The NDTS daemon writes a log file to /opt/nan-dtdaemon/logs/ndtd_audit.txt and can be configured to different levels of verbosity

	* '''nan-dtdaemon~~.log''' — runtime events, heartbeats, errors~~
	* '''ndtd_audit.txt~~''' — one-line summary per experiment~~

	~~This page focuses on nan-dtdaemon.log.~~

	==== '''Log Levels''' ====		==== '''Log Levels''' ====
	Each line begins with a level tag. The level is controlled by the		Each line begins with a level tag. The level is controlled by the
	<code>log_level</code> parameter in <code>ndtd_configuration.dat</code>.		<code>log_level</code> parameter in the configuration file <code>ndtd_configuration.dat</code>.

	{\| class="wikitable"		{\| class="wikitable"
Line 154:		Line 147:

	===== '''Analysis of the Example''' =====		===== '''Analysis of the Example''' =====
	# '''LOG_START''' — is the first line indicating the new instance of the daemon and the data and time that it started.		# '''"LOG_START"''' — is the first line indicating the new instance of the daemon and the data and time that it started.
	# '''"Workstation version is 1.0.15”''' — indicates the daemon is running and the version number		# '''"Workstation version is 1.0.15”''' — indicates the daemon is running and the version number
	# '''“* This is a Topspin Workstation *”''' — indicates that this is a Topspin workstation		# '''“* This is a Topspin Workstation *”''' — indicates that this is a Topspin workstation
Line 161:		Line 154:
	# '''"Workstation user has changed!"''' and next three lines — indicates that the workstation user has changed to nmradmin, that nmradmin is configured to harvest data, and that the harvesting setting is on		# '''"Workstation user has changed!"''' and next three lines — indicates that the workstation user has changed to nmradmin, that nmradmin is configured to harvest data, and that the harvesting setting is on
	# '''“Topspin program has been detected and is running.”''' — daemon detected that the Topspin acquisition directory running		# '''“Topspin program has been detected and is running.”''' — daemon detected that the Topspin acquisition directory running
	# '''“Setting directory to watch …”''' — shows the location of the Topspin directory which the daemon will watch for file modifications that indicate the start and end of an acquisition		# '''“Setting directory to watch to …”''' — shows the location of the Topspin directory which the daemon will watch for file modifications that indicate the start and end of an acquisition

	== '''Troubleshooting Checklist''' ==		== '''Troubleshooting Checklist''' ==

Mmaciejewski at 16:50, 3 June 2025

2025-06-03T16:50:41Z

← Older revision		Revision as of 16:50, 3 June 2025
Line 72:		Line 72:
	* When the Daemon starts it writes the active version number in two palces ...		* When the Daemon starts it writes the active version number in two palces ...
	<pre>/opt/nan-dtdaemon/running_workstation_version-X.Y.Z</pre>which records the version number and a timestamp of when the daemon started		<pre>/opt/nan-dtdaemon/running_workstation_version-X.Y.Z</pre>which records the version number and a timestamp of when the daemon started
	/opt/nan-dtdaemon/logs/nan-dtdaemon.log		<pre>/opt/nan-dtdaemon/logs/nan-dtdaemon.log</pre>
	which write an INFO message stating the version number and which is time-stamped		which write an INFO message stating the version number and which is time-stamped

	=== '''Experiment Transfer Audit File''' ===		=== '''Experiment Transfer Audit File''' ===
	Each time the NDTS daemon processes a completed experiment, it logs a detailed audit entry to the file		Each time the NDTS daemon processes a completed experiment, it logs a detailed audit entry to the file
	<~~code~~>/opt/nan-dtdaemon/logs/ndtd_audit.txt</~~code~~>. This audit trail provides traceable records of all harvesting actions performed by the daemon on a given spectrometer workstation.		<pre>/opt/nan-dtdaemon/logs/ndtd_audit.txt</pre>. This audit trail provides traceable records of all harvesting actions performed by the daemon on a given spectrometer workstation.

	Each line in the audit file includes the following fields:		Each line in the audit file includes the following fields: