Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check on CI conda hasn't "infected" the system #8

Open
mithro opened this issue Jun 21, 2020 · 9 comments
Open

Check on CI conda hasn't "infected" the system #8

mithro opened this issue Jun 21, 2020 · 9 comments
Labels
help wanted Extra attention is needed

Comments

@mithro
Copy link
Owner

mithro commented Jun 21, 2020

The CI system should check that the conda install hasn't done any of the following;

  • Modified the registry
  • Modified the environment (path, etc)
  • Registered shortcuts in the startup menu.
@mithro
Copy link
Owner Author

mithro commented Jun 21, 2020

@xobs / @gojimmypi -- Any idea how I can check this behaviour?

@gojimmypi
Copy link

on an ongoing basis, or just during development of the build process?

to see what's going on at a given time during development, process monitor is pretty helpful. (filters are your friend here)

however during a build process - a "before and after" would probably be quite difficult. In fact, I'm sure the registry could be changing all the time by other windows apps/processes running; how to determine if if was caused by your process?

@mithro
Copy link
Owner Author

mithro commented Jun 21, 2020

@gojimmypi - As part of the CI system.

@mithro
Copy link
Owner Author

mithro commented Jun 21, 2020

We can check just before and after we install conda right?

@xobs
Copy link

xobs commented Jun 22, 2020

Conda appears to create start menu items by unconditionally calling "$INSTDIR\_conda.exe" constructor --prefix "$INSTDIR" --make-menus @MENU_PKGS@:

https://github.com/conda/constructor/blob/master/constructor/nsis/main.nsi.tmpl#L817-L820

They're using NSIS for installation, however they shell out to Python to do the actual start menu creation.

One possibility for determining whether they have updated anything is to use the Volume Shadow Copy service to take a snapshot of the directory both before and after the installation and look for differences. Similar methods could be used to determine which parts of the registry have been modified.

Give me a little bit, I'm putting together a PoC.

GitHub
tool for creating installers from conda packages. Contribute to conda/constructor development by creating an account on GitHub.

@xobs
Copy link

xobs commented Jun 22, 2020

While it's possible to snapshot a user's home directory and compare it both before and after, there are a number of problems:

  1. There doesn't appear to be a way to simply show the delta. You end up having to traverse the entire directory, which takes a while. It may be possible to work around this, but I haven't found a way.
  2. Just like other platforms, you need to be an admin in order to create snapshots. This alone may make it impossible to do in CI
  3. It doesn't cover the registry

The installer doesn't enable logging, but that wouldn't matter because it uses python to create the start menu entries.

@mithro
Copy link
Owner Author

mithro commented Jun 22, 2020

FYI -- We should send a patch upstream to fix the shortcut issue.

This issue is making sure that problems like the shortcut issue don't come back!

@xobs
Copy link

xobs commented Jun 23, 2020

Here's a script that does what you want:

$start_path_global = "$ENV:ProgramData\Microsoft\Windows\Start Menu\Programs"
$start_path_user = "$ENV:AppData\Microsoft\Windows\Start Menu\Programs"
$installer_path = "test/env/downloads/Miniconda3-latest-Windows-x86_64.exe"
$installer_filename = "Miniconda3-latest-Windows-x86_64.exe"
$installer_uri = "https://repo.anaconda.com/miniconda/$installer_filename"
$installer_dest = (Get-Item -Path ".\").FullName + "\tmp"

# Download if necessary
if ( -Not ( Test-Path -Path $installer_path ) ) {
    Write-Output "Installer file not found -- downloading $installer_uri"

    # Create the download directory if it doesn't exist
    $target_dir = [System.IO.Path]::GetDirectoryName($installer_path)
    if ( -Not ( Test-Path -Path $target_dir ) ) {
        New-Item -Path $target_dir -ItemType Directory | Out-Null
    }

    # Perform the download
    Invoke-WebRequest -Uri $installer_uri -OutFile $installer_path
}

# Capture the state of various parts of the system pre-install
Write-Output "Capturing system state prior to installation"
$start_before_global = Get-ChildItem -Recurse -Path $start_path_global
$start_before_user = Get-ChildItem -Recurse -Path $start_path_user
$reg_before_user = Get-ChildItem -Recurse -Path HKCU:\ -ErrorAction SilentlyContinue

# Perform the install
Write-Output "Installing $installer_path to $installer_dest"
Start-Process -Wait -FilePath $installer_path -ArgumentList "/InstallationType=JustMe /AddToPath=0 /RegisterPython=0 /NoRegistry=1 /NoScripts=1 /S /D=$installer_dest"

# Capture the state afterwards
Write-Output "Capturing system state after installation"
$start_after_global = Get-ChildItem -Recurse -Path $start_path_global
$start_after_user = Get-ChildItem -Recurse -Path $start_path_user
$reg_after_user = Get-ChildItem -Recurse -Path HKCU:\ -ErrorAction SilentlyContinue

# Compare the two
Write-Output "Differences before and after:"
Compare-Object -ReferenceObject $start_before_global -DifferenceObject $start_after_global
Compare-Object -ReferenceObject $start_before_user -DifferenceObject $start_after_user
Compare-Object -ReferenceObject $reg_before_user -DifferenceObject $reg_after_user

(Edited to fix installer download)

Example output:

[10:49:47 AM] D:/Code/conda-env-make> .\conda-test-install.ps1
Capturing system state prior to installation
Installing test/env/downloads/Miniconda3-latest-Windows-x86_64.exe to D:\Code\conda-env-make\tmp
Capturing system state after installation
Differences before and after:

InputObject
-----------
C:\Users\smcro\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Anaconda3 (64-bit)
C:\Users\smcro\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Anaconda3 (64-bit)\Anaconda Powershell Prompt (tm…
C:\Users\smcro\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Anaconda3 (64-bit)\Anaconda Prompt (tmp).lnk
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Search\JumplistData

[10:55:17 AM] D:/Code/conda-env-make>

Note how creating Start Menu entries actually modifies the registry slightly by adding the new entries to the JumplistData entry.

This script is rather heavy-handed in that it actually captures the entire registry in a pair of variables, and as such that adds about thirty seconds to the installation time. We could make it fancier and have it check more paths, and/or just loop through an array of paths, but I think this gets the point across.

@mithro
Copy link
Owner Author

mithro commented Jun 23, 2020

30 seconds is entirely fine!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

3 participants