Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests/postgresql-wal-receiver: Port to Python #96349

Merged
merged 1 commit into from Aug 26, 2020

Conversation

dasJ
Copy link
Member

@dasJ dasJ commented Aug 26, 2020

... and remove some weirdnesses.

  • Port to Python
  • Drop the extra pkgs, config, system args
  • Drop all with
  • Use a more standard PostgreSQL data directory
  • Use pkgs and lib from the test runner

Tested with:

  • postgresql_12
  • postgresql_11
  • postgresql_10
  • postgresql_9_6
  • postgresql_9_5

Closes #96347
cc @flokli

Motivation for this change
Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Determined the impact on package closure size (by running nix path-info -S before and after)
  • Ensured that relevant documentation is up to date
  • Fits CONTRIBUTING.md.

@dasJ dasJ requested review from pacien and flokli August 26, 2020 08:55
@dasJ dasJ added 6.topic: testing Tooling for automated testing of packages and modules 8.has: clean-up labels Aug 26, 2020
Copy link
Contributor

@pacien pacien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

restore_command = 'cp ${walBackupDir}/%f %p'
'';
pkg = pkgs."${postgresqlPackage}";
postgresqlDataDir = "/var/db/postgresql/${lib.versions.major pkg.version}";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you forget removing this?

Comment on lines 31 to 33
systemd.tmpfiles.rules = [
"d /var/db/postgresql 0700 postgres postgres"
];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you forget removing this?

services.postgresql = {
package = pkg;
enable = true;
dataDir = postgresqlDataDir;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you forget removing this?

# stop postgres and destroy data
machine.systemctl("stop postgresql")
machine.systemctl("stop postgresql-wal-receiver-main")
machine.succeed("rm -r ${postgresqlDataDir}/{base,global,pg_*}")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be updated.

"for part in ${walBackupDir}/*.partial; do mv $part ''${part%%.*}; done"
) # make use of partial segments too
machine.succeed(
"cp ${recoveryFile}/* ${postgresqlDataDir}/ && chmod 666 ${postgresqlDataDir}/recovery*"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be updated.

$machine->succeed('cp -r ${baseBackupDir}/* ${postgresqlDataDir} && chown postgres:postgres -R ${postgresqlDataDir}');
# restore the base backup
machine.succeed(
"cp -r ${baseBackupDir}/* ${postgresqlDataDir} && chown postgres:postgres -R ${postgresqlDataDir}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be updated.

$machine->waitForUnit('postgresql');
# replay WAL
machine.systemctl("start postgresql")
machine.wait_for_file("${postgresqlDataDir}/recovery.done")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this needs to be updated.

... and remove some weirdnesses.

- Port to Python
- Drop the extra pkgs, config, system args
- Drop all `with`
- Don't override the standard PostgreSQL directory
- Use pkgs and lib from the test runner

Tested with:
- postgresql_12
- postgresql_11
- postgresql_10
- postgresql_9_6
- postgresql_9_5

Closes NixOS#96347
cc @flokli
@dasJ dasJ force-pushed the feat/postgresql-wal-python branch from 29d5125 to ead6de5 Compare August 26, 2020 14:37
@flokli flokli merged commit 36e4ec8 into NixOS:master Aug 26, 2020
@ajs124 ajs124 deleted the feat/postgresql-wal-python branch August 27, 2020 11:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

nixosTests.postgresql-wal-receiver needs to be ported to python
3 participants