Does not reliably reload the configuration

rfc1036 commented

2025-04-03 23:39:45 +02:00

(Migrated from github.com)

Sometime it does, e.g.:

Feb 04 16:31:53 server1 prometheus-sflow-exporter[1342874]: 2025-02-04T15:31:53.193011Z  INFO sflow_exporter: Booting sflow_exporter/0.1.0...
Feb 04 16:31:53 server1 prometheus-sflow-exporter[1342874]: 2025-02-04T15:31:53.193078Z  INFO sflow_exporter: sflow listening at 10.9.9.254:6343/udp...
Feb 04 16:31:53 server1 prometheus-sflow-exporter[1342874]: 2025-02-04T15:31:53.193119Z  INFO sflow_exporter: metrics listening at http://127.0.0.1:9144/metrics...
Feb 04 16:31:53 server1 prometheus-sflow-exporter[1342874]: 2025-02-04T15:31:53.193996Z  INFO sflow_exporter: Loaded 105 routers, 3 agents and 2 ether types
Feb 10 20:00:02 server1 prometheus-sflow-exporter[1342874]: 2025-02-10T19:00:02.732651Z  INFO sflow_exporter: Loaded 104 routers, 3 agents and 2 ether types

But others it does not and I have to restart the daemon. Maybe it does not notice the configuration file being replaced instead of being rewritten?

Sometime it does, e.g.: ``` Feb 04 16:31:53 server1 prometheus-sflow-exporter[1342874]: 2025-02-04T15:31:53.193011Z INFO sflow_exporter: Booting sflow_exporter/0.1.0... Feb 04 16:31:53 server1 prometheus-sflow-exporter[1342874]: 2025-02-04T15:31:53.193078Z INFO sflow_exporter: sflow listening at 10.9.9.254:6343/udp... Feb 04 16:31:53 server1 prometheus-sflow-exporter[1342874]: 2025-02-04T15:31:53.193119Z INFO sflow_exporter: metrics listening at http://127.0.0.1:9144/metrics... Feb 04 16:31:53 server1 prometheus-sflow-exporter[1342874]: 2025-02-04T15:31:53.193996Z INFO sflow_exporter: Loaded 105 routers, 3 agents and 2 ether types Feb 10 20:00:02 server1 prometheus-sflow-exporter[1342874]: 2025-02-10T19:00:02.732651Z INFO sflow_exporter: Loaded 104 routers, 3 agents and 2 ether types ``` But others it does not and I have to restart the daemon. Maybe it does not notice the configuration file being replaced instead of being rewritten?

liske commented

2025-04-04 11:08:43 +02:00

(Migrated from github.com)

inotify is used to watch the config file: github.com/dd-ix/sflow_exporter@4374f8ca79/src/main.rs (L86)

Do you always replace the config file the same way (syscalls) when it does not work reliable?

inotify is used to watch the config file: https://github.com/dd-ix/sflow_exporter/blob/4374f8ca7987419f1cbf46e1648accb7144dccf4/src/main.rs#L86 Do you always replace the config file the same way (syscalls) when it does not work reliable?

rfc1036 commented

2025-04-04 11:25:54 +02:00

(Migrated from github.com)

Yes, the configuration generator basically uses:

use Path::Tiny;

my $yaml = Dump($configuration);
my $file = path('/etc/prometheus/sflow_exporter.yaml');
$file->spew_raw($yaml);

And the spew() documentation says the file is written to a temporary file and then renamed.

When I notice it there is nothing else I can manually do to the file (append to it, move it, copy it...) to make the daemon reload it again.

Yes, the configuration generator basically uses: ```perl use Path::Tiny; my $yaml = Dump($configuration); my $file = path('/etc/prometheus/sflow_exporter.yaml'); $file->spew_raw($yaml); ``` And the `spew()` documentation says _the file is written to a temporary file and then renamed_. When I notice it there is nothing else I can manually do to the file (append to it, move it, copy it...) to make the daemon reload it again.

MarcelCoding commented

2025-04-04 22:26:29 +02:00

(Migrated from github.com)

The problem seemed to be, that the meta file was watched directly instead of the parent directly. According to some testing done by @liske this was wrong.

This should be fixed in v0.1.2.

Watching meta.yaml directly:

Watching parent folder:

The problem seemed to be, that the meta file was watched directly instead of the parent directly. According to some testing done by @liske this was wrong. This should be fixed in [v0.1.2](https://github.com/dd-ix/sflow_exporter/releases/tag/v0.1.2). Watching meta.yaml directly: ![config](https://github.com/user-attachments/assets/24f78d24-1abb-48af-b35e-938cff170861) Watching parent folder: ![parent](https://github.com/user-attachments/assets/36e98124-bc3a-4b49-adc2-4e4259b637b9)

liske commented

2025-04-06 21:54:05 +02:00

(Migrated from github.com)

Although 15c03e4691 fixes the root cause of this issue it adds a minor regression by unnecessarily reloading the configuration whenever any file is written or moved to the configuration directory. IMHO the loop handling the inotify events needs to check if the configuration file was affected at all.

Although 15c03e4691e03628b59f447859e793afe331b00d fixes the root cause of this issue it adds a minor regression by unnecessarily reloading the configuration whenever any file is written or moved to the configuration directory. IMHO [the loop](https://github.com/dd-ix/sflow_exporter/blob/15c03e4691e03628b59f447859e793afe331b00d/src/main.rs#L99-L101) handling the inotify events needs to check if the configuration file was affected at all.

MarcelCoding commented

2025-04-12 01:24:04 +02:00

(Migrated from github.com)

Considering, we just load a yaml and load it into an internal data structure. I would argue that this is not enough compute that it would justify comparing the config to the previews version of the config.

MarcelCoding commented

2025-04-12 19:18:18 +02:00

(Migrated from github.com)

I did not notice that the inotify event also contained the file name of the changed file when watching the parent directory. I've updated the code: github.com/dd-ix/sflow_exporter@e297046ff1

I did not notice that the inotify event also contained the file name of the changed file when watching the parent directory. I've updated the code: https://github.com/dd-ix/sflow_exporter/commit/e297046ff11cd168215113af3c80781da69006cb

❤️ 1

Rows
Columns

Does not reliably reload the configuration #12