Fix `make env-info` on Windows #12

mithro · 2020-06-21T22:10:11Z

The current output of make env-info on Windows looks like this;

$ make env-info
OS_TYPE=Windows CPU_TYPE=x86_64
'name: make-env' 'make-env'
"               Currently running on: 'Windows (x86_64)'"
ECHO is off.
"         Conda environment is named: 'make-env'"
"   Conda Env Top level directory is: 'C:\Users\travis\build\mithro\conda-env-make\test'"
"         Git top level directory is: '$(git rev-parse --show-toplevel)'"
"              The version number is: '$(git describe)'"
"            Git repository is using: $(du -h -s $(git rev-parse --show-toplevel)/.git | sed -e's/\s.*//')" 	
ECHO is off.
"     Environment setup directory is: 'C:\Users\travis\build\mithro\conda-env-make\test\env'" 	
"    Download and cache directory is: 'C:\Users\travis\build\mithro\conda-env-make\test\env\downloads' (using $(du -h -s C:\Users\travis\build\mithro\conda-env-make\test\env\downloads | sed -e's/\s.*//'))" 	
"               Conda's directory is: 'C:\Users\travis\build\mithro\conda-env-make\test\env\conda' (using $(du -h -s C:\Users\travis\build\mithro\conda-env-make\test\env\conda | sed -e's/\s.*//'))" 	
" Conda's packages download cache is: 'C:\Users\travis\build\mithro\conda-env-make\test\env\downloads\conda-pkgs' (using $(du -h -s C:\Users\travis\build\mithro\conda-env-make\test\env\downloads\conda-pkgs | sed -e's/\s.*//'))" 	
"           Conda's Python binary is: 'C:\Users\travis\build\mithro\conda-env-make\test\env\conda\envs\make-env\python.exe'"	
The command "make env-info" exited with 0.

It should look like this;

$ make env-info
OS_TYPE=Linux CPU_TYPE=x86_64
'name: make-env' 'make-env'
               Currently running on: 'Linux (x86_64)'
         Conda environment is named: 'make-env'
   Conda Env Top level directory is: '/home/travis/build/mithro/conda-env-make/test'
         Git top level directory is: '/home/travis/build/mithro/conda-env-make'
              The version number is: 'v0.0-31-gf7d4e6c'
            Git repository is using: 200K
     Environment setup directory is: '$TOP_DIR/env'
    Download and cache directory is: '$TOP_DIR/env/downloads' (using 420M)
               Conda's directory is: '$TOP_DIR/env/conda' (using 571M)
 Conda's packages download cache is: '$TOP_DIR/env/downloads/conda-pkgs' (using 335M)
           Conda's Python binary is: '$TOP_DIR/env/conda/envs/make-env/bin/python'
The command "make env-info" exited with 0.

The text was updated successfully, but these errors were encountered:

gojimmypi · 2020-06-21T23:45:40Z

this is what I see in WSL:

$ make env-info
OS_TYPE=Linux CPU_TYPE=x86_64
'name: make-env' 'make-env'
               Currently running on: 'Linux (x86_64)'

         Conda environment is named: 'make-env'
   Conda Env Top level directory is: '/mnt/c/workspace/make-env/test'
         Git top level directory is: '/mnt/c/workspace/make-env'
              The version number is: 'v0.0-33-gdbba1a2'
            Git repository is using: 73K

     Environment setup directory is: '$TOP_DIR/env'
    Download and cache directory is: '$TOP_DIR/env/downloads' (using 414M)
               Conda's directory is: '$TOP_DIR/env/conda' (using 554M)
 Conda's packages download cache is: '$TOP_DIR/env/downloads/conda-pkgs' (using 329M)
           Conda's Python binary is: '$TOP_DIR/env/conda/envs/make-env/bin/python'

$ make test-command
OS_TYPE=Linux CPU_TYPE=x86_64
'name: make-env' 'make-env'
source /mnt/c/workspace/make-env/test/env/conda/bin/activate make-env && echo "Python is $(which python)"
Python is /mnt/c/workspace/make-env/test/env/conda/envs/make-env/bin/python
source /mnt/c/workspace/make-env/test/env/conda/bin/activate make-env && python --version
Python 3.8.3

heads up I also see this during the first make test-command :

source /mnt/c/workspace/make-env/test/env/conda/bin/activate && conda env update --name make-env --file environment.yml
Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies.  Conda may not use the correct pip to install your packages, and they may end up in the wrong place.  Please add an explicit pip dependency.  I'm adding one for you, but still nagging you.
...
Verifying transaction: /
SafetyError: The package for ncurses located at /mnt/c/workspace/make-env/test/env/downloads/conda-pkgs/ncurses-6.2-he6710b0_1
appears to be corrupted. The path 'share/terminfo/E/Eterm'
has an incorrect size.
  reported size: 2224 bytes
  actual size: 842 bytes

gojimmypi · 2020-06-21T23:54:55Z

From a DOS prompt, I installed the chocolatey make but the Makefile seems to be inaccessible:

From an admin prompt

C:\WINDOWS\system32>choco install make
Chocolatey v0.10.15
Installing the following packages:
make
By installing you accept licenses for the packages.
Progress: Downloading make 4.3... 100%

make v4.3 [Approved]
make package files install completed. Performing other installation steps.
 ShimGen has successfully created a shim for make.exe
 The install of make was successful.
  Software install location not explicitly set, could be in package or
  default install location if installer.

Chocolatey installed 1/1 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

From a regular, non-admin prompt, or admin prompt:

C:\workspace>choco
Chocolatey v0.10.15
Please run 'choco -?' or 'choco <command> -?' for help menu.

C:\workspace>cd C:\workspace\make-env

C:\workspace\make-env>cd test

C:\workspace\make-env\test>make env-info
make: Makefile: Invalid argument
make: *** No rule to make target 'Makefile'.  Stop.

C:\workspace\make-env\test>dir
 Volume in drive C is Windows
 Volume Serial Number is 9078-2015

 Directory of C:\workspace\make-env\test

06/21/2020  04:42 PM    <DIR>          .
06/21/2020  04:42 PM    <DIR>          ..
06/21/2020  04:34 PM                 4 .gitignore
06/21/2020  04:36 PM    <DIR>          env
06/21/2020  04:34 PM               104 environment.yml
06/21/2020  04:34 PM    <JUNCTION>     Makefile [...]
06/21/2020  04:34 PM                44 requirements.txt
06/21/2020  04:34 PM    <DIR>          third_party
               4 File(s)            152 bytes
               4 Dir(s)  42,873,483,264 bytes free

C:\workspace\make-env\test>notepad Makefile

C:\workspace\make-env\test>type MakeFile
The file cannot be accessed by the system.

C:\workspace\make-env\test>

Edit: the above is when this command runs in WSL:

git clone -c core.symlinks=true https://github.com/SymbiFlow/make-env.git

Run the same command in a DOS window, and no Makefile is available in the test directory.

mithro · 2020-06-22T02:21:16Z

@gojimmypi Do you have "developer mode" enabled on Windows?

Could you see if you can figure out how mklink behaves for you?
https://www.howtogeek.com/292914/what-is-developer-mode-in-windows-10/#:~:text=How%20to%20Enable%20Developer%20Mode,be%20put%20into%20Developer%20Mode.

In Developer Mode, a user account with any level of privileges can create symbolic links. In other words, you can open a normal Command Prompt window and use the mklink command. Outside of Developer Mode, you’d need to open a Command Prompt window as Administrator before using the mklink command.

Symbolic links are often used by developers, so this change makes it possible for development tools to create and work with symbolic links without having to run as Administrator.

How-To Geek
What Is “Developer Mode” in Windows 10?
If you dig through Windows 10’s settings, you may come across something called “Developer Mode”. When put into Developer Mode, Windows allows you to more easily test apps you’re developing, use the Ubuntu Bash shell environment, change a variety of developer-focused settings, and do other such things.

mithro · 2020-06-22T02:22:10Z

https://bugsdb.com/_en/debug/0379b1f4365787e6eef9c6d6d8c1d864 might also have more information on what is going on with the symlinks...

What happens when I clone a repository with symlinks on Windows? - BugsDB

gojimmypi · 2020-06-22T17:14:34Z

on the topic of symlinks, this is interesting - particularly regarding relative or absolute links.

I didn't have developer mode turned on. Turning it on, and trying again in either regular or an elevated DOS prompt:

C:\temp\tstenv>git clone -c core.symlinks=true https://github.com/SymbiFlow/make-env.git
Cloning into 'make-env'...
remote: Enumerating objects: 43, done.
remote: Counting objects: 100% (43/43), done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 125 (delta 21), reused 30 (delta 13), pack-reused 82   eiving objects:  72%
Receiving objects: 100% (125/125), 27.02 KiB | 2.70 MiB/s, done.
Resolving deltas: 100% (61/61), done.

C:\temp\tstenv>cd make-env/test

C:\temp\tstenv\make-env\test>make test-command
OS_TYPE=Windows CPU_TYPE=x86_64
'name: make-env' 'make-env'
powershell C:\temp\tstenv\make-env\test\third_party\make-env\mkdir.ps1 "C:\temp\tstenv\make-env\test\env\downloads"
C:\temp\tstenv\make-env\test\third_party\make-env\mkdir.ps1 : File
C:\temp\tstenv\make-env\test\third_party\make-env\mkdir.ps1 cannot be loaded because running scripts is disabled on
this system. For more information, see about_Execution_Policies at https:/go.microsoft.com/fwlink/?LinkID=135170.
At line:1 char:1
+ C:\temp\tstenv\make-env\test\third_party\make-env\mkdir.ps1 C:\temp\t ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : SecurityError: (:) [], PSSecurityException
    + FullyQualifiedErrorId : UnauthorizedAccess
make: *** [third_party/make-env/conda.mk:97: C:\temp\tstenv\make-env\test\env\downloads] Error 1

seems like a lot of security vulnerabilities need to be enabled to make this work? kinda not cool to enable unsigned powershell scripts just for this install. I have many repos over quite a few years, and never needed to do this.

perhaps reconsider symlinks and powershell scripts?

oddly, .\make-env\test\third_party\make-env appears to be a shortcut back to .make-env

fwiw: this works, no error:

make-env>powershell -f touch.ps1 license

I wonder if instead of the powershell script to create directories, just keep it simple with
if not exist ".\wow2" mkdir "wow2"

gojimmypi · 2020-06-22T18:00:17Z

the more I think about this.... why not make the "windows version" simply be in WSL? It would be vastly easier to support.

mithro mentioned this issue Jun 22, 2020

test: Adding pip to environment. #13

Merged

mithro added help wanted Extra attention is needed bug Something isn't working labels Jul 23, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix `make env-info` on Windows #12

Fix `make env-info` on Windows #12

mithro commented Jun 21, 2020

gojimmypi commented Jun 21, 2020

gojimmypi commented Jun 21, 2020 •

edited

mithro commented Jun 22, 2020 •

edited by unfurl-links bot

mithro commented Jun 22, 2020 •

edited by unfurl-links bot

gojimmypi commented Jun 22, 2020

gojimmypi commented Jun 22, 2020

Fix make env-info on Windows #12

Fix make env-info on Windows #12

Comments

mithro commented Jun 21, 2020

gojimmypi commented Jun 21, 2020

gojimmypi commented Jun 21, 2020 • edited

mithro commented Jun 22, 2020 • edited by unfurl-links bot

mithro commented Jun 22, 2020 • edited by unfurl-links bot

gojimmypi commented Jun 22, 2020

gojimmypi commented Jun 22, 2020

Fix `make env-info` on Windows #12

Fix `make env-info` on Windows #12

gojimmypi commented Jun 21, 2020 •

edited

mithro commented Jun 22, 2020 •

edited by unfurl-links bot

mithro commented Jun 22, 2020 •

edited by unfurl-links bot