Urgent Help MSR stopped running
-
I guess MSR is not currently logging as its down. The last line in the log is this at 16:16 pm which was 4 hours ago.
[latest-22053]2022-03-18T16:16:32.655Z InfluxFeed:ERR [InfluxFeed]InfluxFeed#influx write failed (data lost): [Error]Er
And there was lost of these lines:
[latest-22053]2022-03-18T16:16:29.947Z InfluxFeed:ERR [InfluxFeed]InfluxFeed#influx write failed (data lost): [Error]Error: connect ECONNREFUSED 192.168.0.36:8086
But I don't know when the Pi went down? and the InfluxDB so may or may not be related.
-
Troubleshooting section of the manual.
My guess is (a) some hub isn't connecting and you didn't wait long enough for retries to time out (most common, but not only reason), and then you damaged the config file trying to edit out InfluxDB, which won't get in the way, it's just chatty. Follow the steps in the docs. Also, If you installed node-red after MSR, there is a bug in their install script that can corrupt your nodejs install system-wide. Hopefully it's not that. Back on Monday. Good luck!
-
Again, follow the steps in the Troubleshooting section of the docs.
-
Again, follow the steps in the Troubleshooting section of the docs.
I am trying to download the log files from the /var/log folder now.
Is this bad ?
EDIT:
Was in the wrong directory, this looks better.
-
You should not be running npm as root.
-
The syslog is massive and I am not sure what I am looking for, however I just saw this:
Mar 18 18:03:33 HP-Thin01 node[407]: NODE_PATH /home/stuart/reactor Mar 18 18:03:33 HP-Thin01 node[407]: node:internal/fs/utils:344 Mar 18 18:03:33 HP-Thin01 node[407]: throw err; Mar 18 18:03:33 HP-Thin01 node[407]: ^ Mar 18 18:03:33 HP-Thin01 node[407]: Error: ENOSPC: no space left on device, write Mar 18 18:03:33 HP-Thin01 node[407]: at Object.writeSync (node:fs:884:3) Mar 18 18:03:33 HP-Thin01 node[407]: at Object.writeFileSync (node:fs:2159:26) Mar 18 18:03:33 HP-Thin01 node[407]: at Object.<anonymous> (/home/stuart/reactor/app.js:236:133) Mar 18 18:03:33 HP-Thin01 node[407]: at Module._compile (node:internal/modules/cjs/loader:1101:14) Mar 18 18:03:33 HP-Thin01 node[407]: at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) Mar 18 18:03:33 HP-Thin01 node[407]: at Module.load (node:internal/modules/cjs/loader:981:32) Mar 18 18:03:33 HP-Thin01 node[407]: at Function.Module._load (node:internal/modules/cjs/loader:822:12) Mar 18 18:03:33 HP-Thin01 node[407]: at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) Mar 18 18:03:33 HP-Thin01 node[407]: at node:internal/main/run_main_module:17:47 { Mar 18 18:03:33 HP-Thin01 node[407]: errno: -28, Mar 18 18:03:33 HP-Thin01 node[407]: syscall: 'write', Mar 18 18:03:33 HP-Thin01 node[407]: code: 'ENOSPC' Mar 18 18:03:33 HP-Thin01 node[407]: } Mar 18 18:03:33 HP-Thin01 systemd[1]: reactor.service: Main process exited, code=exited, status=1/FAILURE Mar 18 18:03:33 HP-Thin01 systemd[1]: reactor.service: Failed with result 'exit-code'.
If I am running out of disk space, I am not sure really how and where to find out what is talking up all the space and how to clear it.
I am only running MSR, Node-Red and HA Bridge nothing else on this box.
Thanks
-
Ok, so now look at the logs again. Disk space problem hiding others, now you should be able to see them.
-
OK these are the latest entries in the syslog to do with reactor
Mar 18 22:07:48 HP-Thin01 node[522]: NODE_PATH /home/stuart/reactor Mar 18 22:07:48 HP-Thin01 node[522]: FileLogStream { type: 'file', name: 'reactor.log', maxsize: 2, keep: 5 } Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.123Z <app:null> Reactor latest-22069-ebeefad starting on v16.13.1 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.126Z <app:INFO> Process ID 522; platform linux/x64 #1 SMP Debian 4.19.208-1 (2021-09-29); locale en_GB.UTF-8 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.127Z <app:INFO> Basedir /home/stuart/reactor; data in /home/stuart/reactor/storage Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.128Z <app:INFO> NODE_PATH=/home/stuart/reactor Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.135Z <app:INFO> Configured locale (undefined); selected locale(s) en_GB.UTF-8 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.195Z <app:INFO> Loaded locale en-US Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.223Z <Capabilities:null> Module Capabilities v21333 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.354Z <Plugin:null> Module Plugin v21186 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.369Z <TimerBroker:null> Module TimerBroker v21333 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.381Z <default:INFO> Module Entity v22042 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.391Z <Controller:null> Module Controller v22003 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.392Z <default:null> Module Structure v22047 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.413Z <default:null> Module Ruleset v21096 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.414Z <default:null> Module Rulesets v21096 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.433Z <GlobalExpression:null> Module GlobalExpression v21333 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.474Z <Predicate:null> Module Predicate v22067 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.484Z <AlertManager:null> Module AlertManager v21355 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.490Z <default:null> Module Rule v22043 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.500Z <GlobalReaction:null> Module GlobalReaction v21333 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.504Z <default:null> Module Engine v22053 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.505Z <default:null> Module httpapi v22021 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.510Z <default:null> Module httpproxy v21333 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.554Z <default:null> Module wsapi v21334 Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.558Z <app:NOTICE> Starting Structure... ar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.558Z <app:NOTICE> Starting Structure... Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.567Z <IndividualFileStrategy:ERR> [IndividualFileStrategy][IndividualFileStrategy#/home/stuart/reactor/storage] failed to read sys_alerts in /home/stuart/reactor/storage/states/sys_alerts.json: [SyntaxError]SyntaxError: Unexpected end of JSON input Mar 18 22:07:48 HP-Thin01 node[522]: Trapped unhandled Promise rejection SyntaxError: Unexpected end of JSON input Mar 18 22:07:48 HP-Thin01 node[522]: at JSON.parse (<anonymous>) Mar 18 22:07:48 HP-Thin01 node[522]: at IndividualFileStrategy.getDataObject (/home/stuart/reactor/server/lib/IndividualFileStrategy.js:123:47) Mar 18 22:07:48 HP-Thin01 node[522]: at Container.getDataObject (/home/stuart/reactor/server/lib/Container.js:98:51) Mar 18 22:07:48 HP-Thin01 node[522]: at Structure.start (/home/stuart/reactor/server/lib/Structure.js:280:631) Mar 18 22:07:48 HP-Thin01 node[522]: at Object.<anonymous> (/home/stuart/reactor/app.js:288:80) Mar 18 22:07:48 HP-Thin01 node[522]: at Module._compile (node:internal/modules/cjs/loader:1101:14) Mar 18 22:07:48 HP-Thin01 node[522]: at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) Mar 18 22:07:48 HP-Thin01 node[522]: at Module.load (node:internal/modules/cjs/loader:981:32) Mar 18 22:07:48 HP-Thin01 node[522]: at Function.Module._load (node:internal/modules/cjs/loader:822:12) Mar 18 22:07:48 HP-Thin01 node[522]: at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) Mar 18 22:07:48 HP-Thin01 node[522]: SyntaxError: Unexpected end of JSON input Mar 18 22:07:48 HP-Thin01 node[522]: at JSON.parse (<anonymous>) Mar 18 22:07:48 HP-Thin01 node[522]: at IndividualFileStrategy.getDataObject (/home/stuart/reactor/server/lib/IndividualFileStrategy.js:123:47) Mar 18 22:07:48 HP-Thin01 node[522]: at Container.getDataObject (/home/stuart/reactor/server/lib/Container.js:98:51) Mar 18 22:07:48 HP-Thin01 node[522]: at Structure.start (/home/stuart/reactor/server/lib/Structure.js:280:631) Mar 18 22:07:48 HP-Thin01 node[522]: at Object.<anonymous> (/home/stuart/reactor/app.js:288:80) Mar 18 22:07:48 HP-Thin01 node[522]: at Module._compile (node:internal/modules/cjs/loader:1101:14) Mar 18 22:07:48 HP-Thin01 node[522]: at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) Mar 18 22:07:48 HP-Thin01 node[522]: at Module.load (node:internal/modules/cjs/loader:981:32) Mar 18 22:07:48 HP-Thin01 node[522]: at Function.Module._load (node:internal/modules/cjs/loader:822:12) Mar 18 22:07:48 HP-Thin01 node[522]: at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) Mar 18 22:07:48 HP-Thin01 node[522]: Promise { Mar 18 22:07:48 HP-Thin01 node[522]: <rejected> SyntaxError: Unexpected end of JSON input Mar 18 22:07:48 HP-Thin01 node[522]: at JSON.parse (<anonymous>) Mar 18 22:07:48 HP-Thin01 node[522]: at IndividualFileStrategy.getDataObject (/home/stuart/reactor/server/lib/IndividualFileStrategy.js:123:47) Mar 18 22:07:48 HP-Thin01 node[522]: at Container.getDataObject (/home/stuart/reactor/server/lib/Container.js:98:51) Mar 18 22:07:48 HP-Thin01 node[522]: at Structure.start (/home/stuart/reactor/server/lib/Structure.js:280:631) Mar 18 22:07:48 HP-Thin01 node[522]: at Object.<anonymous> (/home/stuart/reactor/app.js:288:80) Mar 18 22:07:48 HP-Thin01 node[522]: at Module._compile (node:internal/modules/cjs/loader:1101:14) Mar 18 22:07:48 HP-Thin01 node[522]: at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) Mar 18 22:07:48 HP-Thin01 node[522]: at Module.load (node:internal/modules/cjs/loader:981:32) Mar 18 22:07:48 HP-Thin01 node[522]: at Function.Module._load (node:internal/modules/cjs/loader:822:12) Mar 18 22:07:48 HP-Thin01 node[522]: at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) Mar 18 22:07:48 HP-Thin01 node[522]: } Mar 18 22:07:48 HP-Thin01 node[522]: Trace Mar 18 22:07:48 HP-Thin01 node[522]: at process.<anonymous> (/home/stuart/reactor/app.js:271:162) Mar 18 22:07:48 HP-Thin01 node[522]: at process.emit (node:events:390:28) Mar 18 22:07:48 HP-Thin01 node[522]: at process.emit (node:domain:475:12) Mar 18 22:07:48 HP-Thin01 node[522]: at emit (node:internal/process/promises:136:22) Mar 18 22:07:48 HP-Thin01 node[522]: at processPromiseRejections (node:internal/process/promises:242:25) Mar 18 22:07:48 HP-Thin01 node[522]: at processTicksAndRejections (node:internal/process/task_queues:97:32) Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.582Z <app:ERR> Trapped unhandled Promise rejection: [SyntaxError]SyntaxError: Unexpected end of JSON input Mar 18 22:07:48 HP-Thin01 node[522]: [latest-22069]2022-03-18T22:07:48.582Z <app:ERR> Please refer to the console log for trace Mar 18 22:07:48 HP-Thin01 systemd[1]: reactor.service: Succeeded.
Mar 18 22:05:28 HP-Thin01 node[402]: [latest-22069]2022-03-18T22:05:28.048Z <IndividualFileStrategy:ERR> [IndividualFileStrategy][IndividualFileStrategy#/home/stuart/reactor/storage] failed to read sys_alerts in /home/stuart/reactor/storage/states/sys_alerts.json: [SyntaxError]SyntaxError: Unexpected end of JSON input Mar 18 22:05:28 HP-Thin01 node[402]: Trapped unhandled Promise rejection SyntaxError: Unexpected end of JSON input Mar 18 22:05:28 HP-Thin01 node[402]: at JSON.parse (<anonymous>) Mar 18 22:05:28 HP-Thin01 node[402]: at IndividualFileStrategy.getDataObject (/home/stuart/reactor/server/lib/IndividualFileStrategy.js:123:47) Mar 18 22:05:28 HP-Thin01 node[402]: at Container.getDataObject (/home/stuart/reactor/server/lib/Container.js:98:51) Mar 18 22:05:28 HP-Thin01 node[402]: at Structure.start (/home/stuart/reactor/server/lib/Structure.js:280:631) Mar 18 22:05:28 HP-Thin01 node[402]: at Object.<anonymous> (/home/stuart/reactor/app.js:288:80) Mar 18 22:05:28 HP-Thin01 node[402]: at Module._compile (node:internal/modules/cjs/loader:1101:14) Mar 18 22:05:28 HP-Thin01 node[402]: at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) Mar 18 22:05:28 HP-Thin01 node[402]: at Module.load (node:internal/modules/cjs/loader:981:32) Mar 18 22:05:28 HP-Thin01 node[402]: at Function.Module._load (node:internal/modules/cjs/loader:822:12) Mar 18 22:05:28 HP-Thin01 node[402]: at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) Mar 18 22:05:28 HP-Thin01 node[402]: SyntaxError: Unexpected end of JSON input Mar 18 22:05:28 HP-Thin01 node[402]: at JSON.parse (<anonymous>) Mar 18 22:05:28 HP-Thin01 node[402]: at IndividualFileStrategy.getDataObject (/home/stuart/reactor/server/lib/IndividualFileStrategy.js:123:47) Mar 18 22:05:28 HP-Thin01 node[402]: at Container.getDataObject (/home/stuart/reactor/server/lib/Container.js:98:51) Mar 18 22:05:28 HP-Thin01 node[402]: at Structure.start (/home/stuart/reactor/server/lib/Structure.js:280:631) Mar 18 22:05:28 HP-Thin01 node[402]: at Object.<anonymous> (/home/stuart/reactor/app.js:288:80) Mar 18 22:05:28 HP-Thin01 node[402]: at Module._compile (node:internal/modules/cjs/loader:1101:14) Mar 18 22:05:28 HP-Thin01 node[402]: at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) Mar 18 22:05:28 HP-Thin01 node[402]: at Module.load (node:internal/modules/cjs/loader:981:32) Mar 18 22:05:28 HP-Thin01 node[402]: at Function.Module._load (node:internal/modules/cjs/loader:822:12) Mar 18 22:05:28 HP-Thin01 node[402]: at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) Mar 18 22:05:28 HP-Thin01 node[402]: Promise { Mar 18 22:05:28 HP-Thin01 node[402]: <rejected> SyntaxError: Unexpected end of JSON input Mar 18 22:05:28 HP-Thin01 node[402]: at JSON.parse (<anonymous>) Mar 18 22:05:28 HP-Thin01 node[402]: at IndividualFileStrategy.getDataObject (/home/stuart/reactor/server/lib/IndividualFileStrategy.js:123:47) Mar 18 22:05:28 HP-Thin01 node[402]: at Container.getDataObject (/home/stuart/reactor/server/lib/Container.js:98:51) Mar 18 22:05:28 HP-Thin01 node[402]: at Structure.start (/home/stuart/reactor/server/lib/Structure.js:280:631) Mar 18 22:05:28 HP-Thin01 node[402]: at Object.<anonymous> (/home/stuart/reactor/app.js:288:80) Mar 18 22:05:28 HP-Thin01 node[402]: at Module._compile (node:internal/modules/cjs/loader:1101:14) Mar 18 22:05:28 HP-Thin01 node[402]: at Object.Module._extensions..js (node:internal/modules/cjs/loader:1153:10) Mar 18 22:05:28 HP-Thin01 node[402]: at Module.load (node:internal/modules/cjs/loader:981:32) Mar 18 22:05:28 HP-Thin01 node[402]: at Function.Module._load (node:internal/modules/cjs/loader:822:12) Mar 18 22:05:28 HP-Thin01 node[402]: at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) Mar 18 22:05:28 HP-Thin01 node[402]: } Mar 18 22:05:28 HP-Thin01 node[402]: Trace Mar 18 22:05:28 HP-Thin01 node[402]: at process.<anonymous> (/home/stuart/reactor/app.js:271:162) Mar 18 22:05:28 HP-Thin01 node[402]: at process.emit (node:events:390:28) Mar 18 22:05:28 HP-Thin01 node[402]: at process.emit (node:domain:475:12) Mar 18 22:05:28 HP-Thin01 node[402]: at emit (node:internal/process/promises:136:22) Mar 18 22:05:28 HP-Thin01 node[402]: at processPromiseRejections (node:internal/process/promises:242:25) Mar 18 22:05:28 HP-Thin01 node[402]: at processTicksAndRejections (node:internal/process/task_queues:97:32) Mar 18 22:05:28 HP-Thin01 node[402]: [latest-22069]2022-03-18T22:05:28.097Z <app:ERR> Trapped unhandled Promise rejection: [SyntaxError]SyntaxError: Unexpected end of JSON input
-
The disk going full has resulted in the corruption of your storage folder. You will need to restore a backup.
-
The disk going full has resulted in the corruption of your storage folder. You will need to restore a backup.
-
@cw-kid it's the folder called
storage
in your Reactor folder.