Updating of edgesense_*.json files stopped

[tanius]

I was able to get generation of the network data packages to work again successfully (see #101 ), however the edgesense_*.json files in /var/www/my-drupal-webroot/sites/example.com/private/edgesense/site_data/ are not regenerated since March 31. Both the JSON files and the directory is writable by the web1 user, used to run Drupal and the cron jobs.

When is updating these files supposed to happen? During Drupal cron run? What could cause this to stop?

[luca]

might this be a problem with permssions? maybe you changed the user that's running the script.
updating this files should happen when the script edgesense_drupal runs i.e. during the cron script run by drush

[albertocottica]

I may have an answer.

I tried opening the views in my browser for research purposes (for example https://edgeryders.eu/en/edgesense_nodes ). users.json and nodes.json have no problem, but the comments.json file times out.

Memory used at: devel_boot()=14.84 MB, devel_shutdown()=252.47 MB, PHP peak=256 MB.

Comments may simply be too big (about 20,000 comments): the restrictions put in place by @tanius on PHP operations on our server block updating after the network becomes large enough.

[luca]

@albertocottica maybe, the current method for generating the jsons on the filesystem to be then processed doesn't pass through the web layer (uses drush + cron), so I'm not sure if this limits apply to that as well

[albertocottica]

Good point. Anyway, I already warned Matthias yesterday.

[tanius]

The problem was this: Edgesense was configured on /admin/edgesense/script-param with "Run during cron: no" and paths (not URLs) were entered into the fields "Users", "Nodes", "Comments" there. With this configuration, when edgesense_drupal is called directly via crontab (not via drush cron or Drupal webcron), it will simply read and parse the existing edgesense_*.json files. They will never be re-generated, since that seems to be part of the Edgesense Drupal cron tasks which we disabled. With URLs, edgesense_drupal would have access to on-the-fly regenerated JSON data, but not with the paths.

To solve this:

• Either do not accept this combination of "Run during cron: no" and paths.
• Or still regenerated the edgesense_*.json files during Drupal cron runs, just letting edgesense_drupal run out of that environment.