metasploit: 4.16.1 -> 5.0.1 #54405

buckley310 · 2019-01-21T06:11:23Z

Motivation for this change

Things done

infinisil · 2019-01-23T23:25:55Z

buckley310 · 2019-01-31T20:33:37Z

At some point since this PR, this expression started failing to build on my workstations, as the hash of "metasploit-framework" in gemset.nix has changed. Since this is being pulled from git, I am unsure why this is.

I originally followed the instructions from "default.nix" when updating, and stripped the dependencies from gemset.nix accordingly. However, since the expression for version 4.16.1 does not seem to strip these dependencies, this is my suspect.

I have rebuilt gemset.nix with dependencies included, to match the previous version of the package, and I have updated the comments in default.nix to match.

ryantm · 2019-02-25T02:23:59Z

@GrahamcOfBorg build metasploit

ryantm · 2019-02-25T03:06:26Z

The build failed for me on NixOS:

$ nix build --no-link --keep-going --max-jobs 4 --option build-use-sandbox true -f /home/ryantm/.cache/nix-review/pr-54405/build.nix
fixed-output derivation produced path '/nix/store/6z5iqdgw0bxwr70rmkk1pg9y1wzzdvhh-metasploit-framework-1442130' with sha256 hash '1ixqx4k2ak2dc912crcg8bq64aqkxn2bjnwcv6wphj8vvyva9z87' instead of the expected hash '0f8kvf5wn16jir9nka1v2jbh2znf7yjg7z7znw026rkwnc52ff82'
cannot build derivation '/nix/store/zq0qk6zsgbdv5hzv539ia7lr28i41dr8-ruby2.5.3-metasploit-framework-5.0.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/0299vyrx6rf16127l8fqjc4jq6zm6f3i-metasploit-bundler-env.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/1hj50ddp089sxa6fqcy1b7jlc6knqznr-metasploit-framework-5.0.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/0v9ysw5z7zd7n9idra4k775x1fwdy3gm-env.drv': 1 dependencies couldn't be built

ryantm

Get it to build

worldofpeace · 2019-02-27T02:16:17Z

cc @alyssais @manveru

@buckley310 They may be able to help you with this.

buckley310 · 2019-02-27T03:27:10Z

Thanks. the issue I am running into is that the hash for "metasploit-framework" in "gemset.nix" keeps changing. It has done so twice now. I am not sure why this is, or how to proceed.

manveru · 2019-02-27T09:07:21Z

The metasploit-framework is fetched from git. So if the revision you build changes, this will change as well. Right now you use the refs/tags/5.0.1, but it might be better to use a fixed revision instead of a tag.

buckley310 · 2019-02-28T15:35:07Z

This package is, once again, building properly. The files that are not auto-generated now reference a specific revision, rather than a git tag.

ryantm · 2019-03-01T05:34:25Z

@GrahamcOfBorg build metasploit

ryantm · 2019-03-01T05:43:21Z

Well darn, it still doesn't build.

$ nix-review pr 54405
$ git fetch --force https://github.com/NixOS/nixpkgs master:refs/nix-review/0 pull/54405/head:refs/nix-review/1
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Total 28 (delta 19), reused 19 (delta 19), pack-reused 9
Unpacking objects: 100% (28/28), done.
From https://github.com/NixOS/nixpkgs
   934b2a8a590..48ed8d4f81e  master               -> refs/nix-review/0
 + 51b54e39481...cd0d315495e refs/pull/54405/head -> refs/nix-review/1  (forced update)
$ git worktree add /home/ryantm/.cache/nix-review/pr-54405/nixpkgs 48ed8d4f81e1f5166299fa0829b83e5a43d78f62
Preparing worktree (detached HEAD 48ed8d4f81e)
Checking out files: 100% (18478/18478), done.
HEAD is now at 48ed8d4f81e lollypop: 0.9.921 -> 0.9.923
$ git merge --no-commit cd0d315495e84980905ebfd4c3347663469fa70d
Automatic merge went well; stopped before committing as requested
$ nix build --no-link --keep-going --max-jobs 4 --option build-use-sandbox true -f /home/ryantm/.cache/nix-review/pr-54405/build.nix
fixed-output derivation produced path '/nix/store/6x1wwf6kdgh3za2s91lr03i6lmy6713n-metasploit-framework-bf949b7' with sha256 hash '0shlc8yq8kfpwa6mk9ns87irl72hgfkgs1qhm34vyak3x99jxc5a' instead of the expected hash '0p5mfrlbl62vi0yf4a4kc0q4nq5ag9kz94jwa1lqqcb1rw7474gk'
cannot build derivation '/nix/store/il7ddkqa9w9wfip88n4v320hzxjzw2zk-ruby2.5.3-metasploit-framework-5.0.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/cf89774139gk06jmwz6mh6vqjssmbiyi-metasploit-bundler-env.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/8a5647d6k4drv6q0bibmbyl1fx8x7wrr-metasploit-framework-5.0.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/23by1bbmj656pdqhl6qc4n19sl5qzb6y-env.drv': 1 dependencies couldn't be built
[22 built (1 failed), 0.0 MiB DL]
error: build of '/nix/store/23by1bbmj656pdqhl6qc4n19sl5qzb6y-env.drv' failed
https://github.com/NixOS/nixpkgs/pull/54405
1 package failed to build:
metasploit

[0.0 MiB DL]
error: build log of '/nix/store/8a5647d6k4drv6q0bibmbyl1fx8x7wrr-metasploit-framework-5.0.1.drv' is not available
No packages were successfully build, skip nix-shell
$ git worktree prune

buckley310 · 2019-03-04T03:41:23Z

I ran a GC on my machine and updated the hash again, in case this was some kind of stale cache issue on my PC. At this point I expect it to break again, but you never know. If it breaks again, it may make sense to redo the PR with a newer version anyway, preferably after root-causing this.

manveru · 2019-03-04T08:45:10Z

Yeah, at this point I'm really not sure what could cause this.

buckley310 · 2019-03-05T21:53:45Z

Yep, broken again.

buckley310 · 2019-03-06T19:49:22Z

It looks like whatever is pulling down the git repo is doing it wrong. Check this out, release 4.17.44 is newer than 5.0.1.

/nix/store/6vxwr49ihrzlk5ljfi93qv0vzmv5i1lq-metasploit-framework-bf949b7 $ git show 4.17.44 | head
tag 4.17.44
Tagger: Metasploit <metasploit@rapid7.com>
Date:   Thu Feb 28 10:03:05 2019 -0800

4.17.44
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAABCAAGBQJceCJZAAoJEM37X6UgB7lU4L0P/176qEST6CJ4oUG17VKmORzT
GdXfoeevbGLBYEFdeDmT2Z3xOGB9FZMW8DKZkrKxOhYjZ7WLZbbmN83u1FxbZa1Z

buckley310 · 2019-03-10T22:47:34Z

The hash has changed again. I saved the derivation from before, and indeed, the contents of the .git folder change. So it is leaving git metadata behind in the folder that it probably shouldn't.

alyssais · 2019-03-12T12:27:35Z

the contents of the .git folder change

What changed?

buckley310 · 2019-03-12T13:43:58Z

This is a diff between a backup from a few days ago, and how it builds now.

$ diff --recursive /scratch/6vxwr49ihrzlk5ljfi93qv0vzmv5i1lq-metasploit-framework-bf949b7/ /nix/store/7805bkk6f9zzwgbh6c2izyj8vqmk03n8-metasploit-framework-bf949b7
diff --recursive /scratch/6vxwr49ihrzlk5ljfi93qv0vzmv5i1lq-metasploit-framework-bf949b7/.git/info/refs /nix/store/7805bkk6f9zzwgbh6c2izyj8vqmk03n8-metasploit-framework-bf949b7/.git/info/refs
807a808,809
> f70115edc945da8a166dfc9a35d1c1f3552ee364	refs/tags/4.17.45
> a5bcabc9c00d70f48535cbed568ccd07dcbfbbfe	refs/tags/4.17.45^{}
diff --recursive /scratch/6vxwr49ihrzlk5ljfi93qv0vzmv5i1lq-metasploit-framework-bf949b7/.git/objects/info/packs /nix/store/7805bkk6f9zzwgbh6c2izyj8vqmk03n8-metasploit-framework-bf949b7/.git/objects/info/packs
1,2c1,2
< P pack-4b102fcb3a3b77edf3b6909b5fb5082cdc658d7c.pack
< P pack-f67758fa719935ceb74604fb0b466e4723fb6a36.pack
---
> P pack-331da3af79bdc054f4c20073fde61d8a1db3b259.pack
> P pack-0fc120ade3ef1dc44dc13d264dee1ee8b5dc5ffb.pack
Only in /nix/store/7805bkk6f9zzwgbh6c2izyj8vqmk03n8-metasploit-framework-bf949b7/.git/objects/pack: pack-0fc120ade3ef1dc44dc13d264dee1ee8b5dc5ffb.idx
Only in /nix/store/7805bkk6f9zzwgbh6c2izyj8vqmk03n8-metasploit-framework-bf949b7/.git/objects/pack: pack-0fc120ade3ef1dc44dc13d264dee1ee8b5dc5ffb.pack
Only in /scratch/6vxwr49ihrzlk5ljfi93qv0vzmv5i1lq-metasploit-framework-bf949b7/.git/objects/pack: pack-f67758fa719935ceb74604fb0b466e4723fb6a36.idx
Only in /scratch/6vxwr49ihrzlk5ljfi93qv0vzmv5i1lq-metasploit-framework-bf949b7/.git/objects/pack: pack-f67758fa719935ceb74604fb0b466e4723fb6a36.pack
diff --recursive /scratch/6vxwr49ihrzlk5ljfi93qv0vzmv5i1lq-metasploit-framework-bf949b7/.git/packed-refs /nix/store/7805bkk6f9zzwgbh6c2izyj8vqmk03n8-metasploit-framework-bf949b7/.git/packed-refs
808a809,810
> f70115edc945da8a166dfc9a35d1c1f3552ee364 refs/tags/4.17.45
> ^a5bcabc9c00d70f48535cbed568ccd07dcbfbbfe

alyssais · 2019-03-17T13:42:06Z

Only thing I can suggest here is having a look at the contents of the packfiles with git-verify-pack to see if you can figure out what's changed.

buckley310 · 2019-03-18T23:12:35Z

The git data actually knows about newer branches than 5.0.1.

alyssais · 2019-03-20T22:43:24Z

The git data actually knows about newer branches than 5.0.1.

Perhaps using the --single-branch argument to git clone would help.

manveru · 2019-04-05T10:57:57Z

So, I looked into the issue again, and have a potential fix at nix-community/bundix#51
I tried building with this locally, and it seems fine. Please let me know if that helps.

buckley310 · 2019-04-18T00:25:01Z

Not working on my machine :\ blunder seems to require ".git" to exist currently.

buckley310 · 2019-06-12T04:16:20Z

This request isn't going anywhere, I do not have the knowledge to debug the build system, and this version is outdated anyhow.

metasploit: 4.16.1 -> 5.0.1

ad23787

GrahamcOfBorg added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Jan 21, 2019

metasploit: Update comments and regenerate gemset.nix

2f10744

ryantm requested changes Feb 25, 2019

View reviewed changes

do not reference revision by tag

cd0d315

update metasploit hash in gemset.nix

9022ad7

infinisil added the 8.has: package (update) label Mar 30, 2019

manveru mentioned this pull request May 3, 2019

[Draft] Ruby vulnerability roundup 001 #58823

Open

27 tasks

buckley310 closed this Jun 12, 2019

buckley310 mentioned this pull request Sep 5, 2019

metasploit: 4.16.1 -> 5.0.45 #68122

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

metasploit: 4.16.1 -> 5.0.1 #54405

metasploit: 4.16.1 -> 5.0.1 #54405

buckley310 commented Jan 21, 2019

infinisil commented Jan 23, 2019

buckley310 commented Jan 31, 2019

ryantm commented Feb 25, 2019

ryantm commented Feb 25, 2019

ryantm left a comment

worldofpeace commented Feb 27, 2019

buckley310 commented Feb 27, 2019

manveru commented Feb 27, 2019

buckley310 commented Feb 28, 2019

ryantm commented Mar 1, 2019

ryantm commented Mar 1, 2019

buckley310 commented Mar 4, 2019

manveru commented Mar 4, 2019

buckley310 commented Mar 5, 2019

buckley310 commented Mar 6, 2019

buckley310 commented Mar 10, 2019

alyssais commented Mar 12, 2019

buckley310 commented Mar 12, 2019

alyssais commented Mar 17, 2019 via email

buckley310 commented Mar 18, 2019

alyssais commented Mar 20, 2019 via email

manveru commented Apr 5, 2019

buckley310 commented Apr 18, 2019

buckley310 commented Jun 12, 2019

metasploit: 4.16.1 -> 5.0.1 #54405

metasploit: 4.16.1 -> 5.0.1 #54405

Conversation

buckley310 commented Jan 21, 2019

Motivation for this change

Things done

infinisil commented Jan 23, 2019

buckley310 commented Jan 31, 2019

ryantm commented Feb 25, 2019

ryantm commented Feb 25, 2019

ryantm left a comment

Choose a reason for hiding this comment

worldofpeace commented Feb 27, 2019

buckley310 commented Feb 27, 2019

manveru commented Feb 27, 2019

buckley310 commented Feb 28, 2019

ryantm commented Mar 1, 2019

ryantm commented Mar 1, 2019

buckley310 commented Mar 4, 2019

manveru commented Mar 4, 2019

buckley310 commented Mar 5, 2019

buckley310 commented Mar 6, 2019

buckley310 commented Mar 10, 2019

alyssais commented Mar 12, 2019

buckley310 commented Mar 12, 2019

alyssais commented Mar 17, 2019 via email

buckley310 commented Mar 18, 2019

alyssais commented Mar 20, 2019 via email

manveru commented Apr 5, 2019

buckley310 commented Apr 18, 2019

buckley310 commented Jun 12, 2019