Preserve remotes when publishing database changes

README.md

@@ -4,7 +4,7 @@
 
 It has much less columns (and removes redundant/unnecessary columns) and is easily readable.
 
-It adds also some helpful columns (`is_organic`, `is_healthy` columns) thanks the [Data Parser's Algorithm](https://github.com/Lifyzer/Data-Parser-System).
+The schema includes `is_organic` and `is_healthy` columns from the [Data Parser project](https://github.com/Lifyzer/Data-Parser-System). In this export, INSERT statements supply `is_healthy` but omit `is_organic`, leaving its default `0`; that default does not establish whether a product is organic.
 
 Finally, the **Lifyzer Database** is available under SQL format (unlike OpenFoodFacts DB) and is split into very small files (to improve readability in those files). 
 
@@ -14,6 +14,34 @@ Finally, the **Lifyzer Database** is available under SQL format (unlike OpenFood
 June 14, 2019  <!-- Update it each time a newer OpenFoodFacts DB has been used -->
 
 
+## Using this historical snapshot
+
+- `MySQL/` contains 815 SQL chunks with 818,846 INSERT statements. Only `food-database-0-1000.sql` creates the `product` table; other chunks depend on it. File ranges are not a guarantee of complete coverage of the upstream dataset.
+- This is the June 2019 snapshot, not a live food database. Existing barcode uniqueness, source attribution and the ODbL licence are preserved.
+- The export is not compatible with current MySQL strict defaults as-is: the first chunk fails on an empty string in the numeric `dietary_fiber` column under MySQL 8.4.11. Other numeric columns also contain empty strings. Missing nutrition values need an explicit policy before migrating the data; converting missing values to zero would change their meaning.
+- A diagnostic import with strict mode disabled still stops on a duplicate barcode (`0019320001376`). Resolve duplicate-record conflicts explicitly; disabling strict mode is not an import solution.
+- The schema uses MySQL `utf8` rather than `utf8mb4`. Character coverage and field lengths also need checking during a migration.
+- `is_healthy` is a historical derived flag, not a validated assessment of a product or an individual's dietary needs.
+
+Keep this source export unchanged when experimenting. Import only into a disposable database until the schema, missing-value policy and resulting row counts have been validated. This repository currently has no automatic data migration or full import test suite.
+
+## Publishing repository changes
+
+`save-project.sh` publishes the current clean branch without changing `origin`. By default it uses the historical Bitbucket, GitLab and GitHub destinations listed in the script; their current availability must be verified separately. An explicit destination limits the run:
+
+```bash
+bash save-project.sh origin
+```
+
+Use a review branch where the destination requires a pull request. The script stops at the first failed push and never force-pushes; earlier successful destinations are not rolled back. It does not commit files or publish uncommitted work.
+
+Run the offline helper checks (no network or credentials required):
+
+```bash
+bash -n save-project.sh
+PYTHONDONTWRITEBYTECODE=1 python3 -m unittest -v test_save_project
+```
+
 ## Who Did This...? 😉
 
 Made with LOT of ❤️ by **[Pierre-Henry Soria](http://pierrehenry.be)**! A passionate software engineer (and in great health **thanks [Lifyzer App](https://lifyzer.com)**! 😸)

save-project.sh

@@ -1,17 +1,27 @@
 #!/bin/bash
+set -euo pipefail
 
-function save-project-to-repo() {
-    git remote rm origin
-    git remote add origin $1
-    git push
+cd -- "$(dirname -- "${BASH_SOURCE[0]}")"
+branch=$(git symbolic-ref --quiet --short HEAD) || {
+    echo 'Check out the intended branch before publishing.' >&2
+    exit 1
 }
+if [[ -n $(git status --porcelain) ]]; then
+    echo 'Review and commit intended changes before publishing.' >&2
+    exit 1
+fi
 
-declare readonly gitRemotes=(
-    git@bitbucket.org:pH_7/lifyzer-database.git
-    git@gitlab.com:pH-7/lifyzer-database.git
-    git@github.com:Lifyzer/Lifyzer-Database.git
-)
-for remote in "${gitRemotes[@]}"
-do
-    save-project-to-repo $remote
+# Optional arguments select explicit destinations for this run.
+# Historical mirrors remain the defaults; no remote configuration is changed.
+if (( $# )); then
+    gitRemotes=("$@")
+else
+    gitRemotes=(
+        git@bitbucket.org:pH_7/lifyzer-database.git
+        git@gitlab.com:pH-7/lifyzer-database.git
+        git@github.com:Lifyzer/Lifyzer-Database.git
+    )
+fi
+for remote in "${gitRemotes[@]}"; do
+    git push -- "$remote" "HEAD:refs/heads/$branch"
 done

test_save_project.py

@@ -0,0 +1,74 @@
+"""Offline publishing-helper checks; fake Git cannot access any remote."""
+import json
+import os
+from pathlib import Path
+import shutil
+import subprocess
+import tempfile
+import unittest
+
+
+class PublishScriptTests(unittest.TestCase):
+    def run_script(self, args=(), *, dirty=False, detached=False, fail_push=False):
+        with tempfile.TemporaryDirectory(prefix='database publisher ') as directory:
+            root = Path(directory)
+            shutil.copy2(Path(__file__).with_name('save-project.sh'), root / 'save-project.sh')
+            fake = root / 'git'
+            fake.write_text('''#!/usr/bin/env python3
+import json, os, sys
+with open(os.environ['GIT_LOG'], 'a') as log:
+    log.write(json.dumps(sys.argv[1:]) + '\\n')
+command = sys.argv[1]
+if command == 'symbolic-ref':
+    if os.environ['DETACHED'] == '1': sys.exit(1)
+    print('codex/reviewed-data')
+elif command == 'status':
+    if os.environ['DIRTY'] == '1': print(' M intentional.sql')
+elif command == 'push':
+    if os.environ['FAIL_PUSH'] == '1': sys.exit(7)
+else:
+    sys.exit('Unexpected or destructive Git command')
+''')
+            fake.chmod(0o755)
+            log = root / 'calls.jsonl'
+            env = dict(os.environ, PATH=f'{root}:{os.environ["PATH"]}', GIT_LOG=str(log),
+                       DIRTY=str(int(dirty)), DETACHED=str(int(detached)), FAIL_PUSH=str(int(fail_push)))
+            result = subprocess.run(['bash', str(root / 'save-project.sh'), *args], env=env,
+                                    cwd='/', capture_output=True, text=True)
+            calls = [json.loads(line) for line in log.read_text().splitlines()]
+            return result, calls
+
+    def test_explicit_destinations_keep_branch_and_remote_configuration(self):
+        result, calls = self.run_script(('origin', '/tmp/mirror with spaces.git'))
+        self.assertEqual(result.returncode, 0, result.stderr)
+        self.assertEqual([c for c in calls if c[0] == 'push'], [
+            ['push', '--', 'origin', 'HEAD:refs/heads/codex/reviewed-data'],
+            ['push', '--', '/tmp/mirror with spaces.git', 'HEAD:refs/heads/codex/reviewed-data']])
+        self.assertFalse(any(c[0] == 'remote' for c in calls))
+
+    def test_historical_defaults_remain_in_order(self):
+        result, calls = self.run_script()
+        self.assertEqual(result.returncode, 0)
+        self.assertEqual([c[2] for c in calls if c[0] == 'push'], [
+            'git@bitbucket.org:pH_7/lifyzer-database.git',
+            'git@gitlab.com:pH-7/lifyzer-database.git',
+            'git@github.com:Lifyzer/Lifyzer-Database.git'])
+
+    def test_dirty_work_is_not_published(self):
+        result, calls = self.run_script(dirty=True)
+        self.assertNotEqual(result.returncode, 0)
+        self.assertFalse(any(c[0] == 'push' for c in calls))
+
+    def test_detached_head_is_not_published(self):
+        result, calls = self.run_script(detached=True)
+        self.assertNotEqual(result.returncode, 0)
+        self.assertFalse(any(c[0] == 'push' for c in calls))
+
+    def test_first_failed_push_stops_later_destinations(self):
+        result, calls = self.run_script(('first', 'second'), fail_push=True)
+        self.assertEqual(result.returncode, 7)
+        self.assertEqual(len([c for c in calls if c[0] == 'push']), 1)
+
+
+if __name__ == '__main__':
+    unittest.main()