Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Third party dir not excluded by license check #17

Closed
acomodi opened this issue Apr 15, 2021 · 11 comments · Fixed by #20
Closed

Third party dir not excluded by license check #17

acomodi opened this issue Apr 15, 2021 · 11 comments · Fixed by #20

Comments

@acomodi
Copy link
Contributor

acomodi commented Apr 15, 2021

I wanted to use this check in GH actions with Symbiflow/actions/checks@main and got this error:

Run SymbiFlow/actions/checks@main
/usr/bin/docker run --name e42b03e80629894ebdbfda1e7e67e6c79d_d603ec --label 5588e4 --workdir /github/workspace --rm -e pythonLocation -e LD_LIBRARY_PATH -e INPUT_LICENSEEXCLUDE -e INPUT_PYTHONEXCLUDE -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/fpga-interchange-tests/fpga-interchange-tests":"/github/workspace" 5588e4:2b03e80629894ebdbfda1e7e67e6c79d

===========================
Check SPDX identifier
===========================

Checking ./Makefile
Checking ./env.sh
Checking ./tests/lut/lut.v
Checking ./third_party/RapidWright/Makefile
Checking ./third_party/RapidWright/interchange/Makefile
Checking ./third_party/RapidWright/scripts/invoke_rapidwright.sh
Checking ./third_party/fpga-interchange-schema/docs/Makefile
Checking ./third_party/fpga-interchange-schema/docs/conf.py
Checking ./third_party/make-env/test/check.py
ERROR: ./Makefile does not have license information.
ERROR: ./env.sh does not have license information.
ERROR: ./tests/lut/lut.v does not have license information.
ERROR: ./third_party/RapidWright/Makefile does not have license information.
ERROR: ./third_party/RapidWright/interchange/Makefile does not have license information.
ERROR: ./third_party/RapidWright/scripts/invoke_rapidwright.sh does not have license information.
ERROR: ./third_party/make-env/test/check.py does not have license information.Run SymbiFlow/actions/checks@main

Apart from the correct first three errors, all the other ones should not happen as they are in third_party directories.
@acomodi
Copy link
Contributor Author

acomodi commented Apr 15, 2021

For reference, this is the failing CI

@acomodi
Copy link
Contributor Author

acomodi commented Apr 16, 2021 •
edited

cc @mithro @umarcor

I am not sure how to grab the exclude lists from the action.yml and pass them to the script. I believe that the issue here is that those exclusion list are actually empty, hence the error.

@umarcor
Copy link
Contributor

umarcor commented Apr 17, 2021 •
edited

@acomodi the arguments/options defined in action.yml are exposed inside the Action environment (the container) as environment variables. So, licenseExclude (https://github.com/SymbiFlow/actions/blob/main/checks/action.yml) is INPUT_LICENSEEXCLUDE (https://github.com/antmicro/fpga-interchange-tests/runs/2354117460?check_suite_focus=true#step:5:10). In the check script, that's used in https://github.com/SymbiFlow/actions/blob/main/checks/check_license.sh#L20-L22.

@acomodi
Copy link
Contributor Author

acomodi commented Apr 17, 2021 •
edited

@umarcor all right, thanks for the explanation. I wonder why the third parties repos are not excluded and the check is performed on them as well

From the action run, this is the setup of the check action:

  with:
    licenseExclude: */.*/* */build/* */env/* */src/* */third_party/* */*/__init__.py ./miniconda.sh
  
    pythonExclude: */.*/* */build/* */env/* */src/* */third_party/* */*/__init__.py ./miniconda.sh
  
  env:
    pythonLocation: /opt/hostedtoolcache/Python/3.9.4/x64
    LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.4/x64/lib

Might it be the ordering of the call to the various gh actions in the workflow?

Additionally, how one would add more items to the exclusion list?

@mithro
Copy link
Contributor

mithro commented Apr 18, 2021

@acomodi - It does seem the exclusion lists are not being connected to the scripts.

@mithro
Copy link
Contributor

mithro commented Apr 18, 2021

The docker command seems to be run using;

/usr/bin/docker run --name e448b33bfa051758d248aca85c1647e3e2_3b18b0 --label 5588e4 --workdir /github/workspace --rm -e INPUT_PYTHONEXCLUDE -e INPUT_LICENSEEXCLUDE -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/actions/actions":"/github/workspace" 5588e4:48b33bfa051758d248aca85c1647e3e2

Which means the INPUT_LICENSEEXCLUDE environment value is unlikely getting into the docker container?

@mithro
Copy link
Contributor

mithro commented Apr 18, 2021

Oh, I'm confused the -e INPUT_PYTHONEXCLUDE is right there.... I'm guessing it isn't set -x so isn't getting through the top level checks.sh script. Checking now....

@mithro
Copy link
Contributor

mithro commented Apr 18, 2021

Actually it just seems to be that shell scripts are bad and we should should stop writing them. I'll probably attempt #21 tomorrow.

@mithro
Copy link
Contributor

mithro commented Apr 18, 2021

The pull request at #20 tests the exclude feature and shows it doesn't currently work.

@umarcor
Copy link
Contributor

umarcor commented Apr 18, 2021

@mithro, when -e ENVVAR_NAME is passed to docker run, it means "expose this envvar inside the container using the same name and content". You don't need to set -e ENVVAR_NAME?"$ENVVAR_NAME" explicitly.

@mithro mithro mentioned this issue Apr 19, 2021
Merged
@mithro
Copy link
Contributor

mithro commented Apr 19, 2021

This will be fixed by #20.

@mithro mithro linked a pull request Apr 19, 2021 that will close this issue
Large rework the checks action #20
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Large rework the checks action mithro/actions
3 participants
@mithro @umarcor @acomodi