ffmpeg{_3_4,_4,-full}: updates #52444

dtzWill · 2018-12-17T16:05:11Z

Motivation for this change

Commits are on top of just-submitted libaom fix PR,
since without that fix newer ffmpeg builds break.

Didn't find changelogs that weren't git commit logs, FWIW.

Things done

dtzWill · 2018-12-19T01:12:51Z

(this should be rebased on top of whatever we end up merging re:libaom over in #52443)

ivan · 2018-12-19T05:23:33Z

git am doesn't seem to like 🠞, it drops the symbol and everything that follows.

dtzWill · 2018-12-19T14:00:35Z

Works for me? (not sure it matters but using git 2.20.1)

Tried using git am on patch generated with git format-patch as well as by github via this URL:
https://github.com/NixOS/nixpkgs/pull/52444/commits/f773cb2e4d82c005fc2476d914dada7f5d01f7eb.patch

However sounds like these cute arrows aren't worth the trouble, was hoping it was a solved problem by now :(.

(and they look terrible in the patch text? eep)

ivan · 2018-12-19T14:05:08Z

Ah, you are correct, I was actually seeing a bug in tig 2.2's rendering of commits in the commit list.

dtzWill · 2018-12-19T14:12:39Z

Okay, well glad to hear it! Dropped them anyway to be safe and potentially easier to read in whatever tooling folks prefer 👍

https://www.ffmpeg.org/download.html#releases https://git.ffmpeg.org/gitweb/ffmpeg.git/shortlog/n4.0.3

dtzWill · 2018-12-28T19:07:05Z

@GrahamcOfBorg build ffmpeg

dtzWill · 2018-12-29T20:05:00Z

Good to merge, any objections/thoughts?

infinisil · 2018-12-29T20:09:10Z

The darwin build seems to be failing?

infinisil · 2018-12-29T20:52:58Z

Well I built ffmpeg_3_4 on Darwin myself and it built fine.

Something else I just noticed, you have ffmpeg: 4.0.2 -> 4.0.3 in the commits, but ffmpeg = ffmpeg_3 = ffmpeg_3_4 by default. ffmpeg-full however is a 4.* version, which is a bit odd.

Edit: Yeah I should've just read ofborg's output a bit better, the ffmpeg-full build does indeed fail for me with the same error.

dtzWill · 2018-12-30T00:06:37Z

Ruhroh, that looks like it was introduced with #52443 sorry! Investigating... 🕵️‍♂️

dtzWill · 2018-12-30T00:34:58Z

@GrahamcOfBorg build libaom

dtzWill · 2018-12-30T00:55:20Z

@GrahamcOfBorg build ffmpeg_4

dtzWill · 2018-12-30T01:54:08Z

Hrmm, not sure how to view (or trigger?) darwin builds-- wanted to see if bumping helped the issue...

infinisil · 2018-12-30T04:20:58Z

I can test it on my own Mac, aom build is failing:

Scanning dependencies of target test_aom_common
[ 72%] Building C object CMakeFiles/set_maps.dir/examples/set_maps.c.o
[ 72%] Building C object CMakeFiles/scalable_encoder.dir/examples/scalable_encoder.c.o
[ 72%] Building C object CMakeFiles/lightfield_bitstream_parsing.dir/examples/lightfield_bitstream_parsing.c.o
[ 72%] Building CXX object CMakeFiles/test_aom_common.dir/test/aom_integer_test.cc.o
/tmp/nix-build-libaom-1.0.0.errata.1.drv-0/aom/examples/lightfield_bitstream_parsing.c
:
393
:
31
:

warning
:
implicit conversion loses integer precision: 'long long' to 'int' [-Wshorten-64-to-32]

  int num_tile_lists = tl_pts - pts;

      ~~~~~~~~~~~~~~   ~~~~~~~^~~~~

[ 73%] Linking CXX executable examples/scalable_encoder
[ 73%] Linking CXX executable examples/set_maps
1 warning generated.
[ 73%] Linking CXX executable examples/lightfield_bitstream_parsing
Undefined symbols for architecture x86_64:
  "_aom_read_obu_header", referenced from:
      _obudec_read_obu_header_and_size in obudec.c.o
  "_aom_wb_write_literal", referenced from:
      _process_tile_list in lightfield_bitstream_parsing.c.o
  "_aom_wb_write_unsigned_literal", referenced from:
      _process_tile_list in lightfield_bitstream_parsing.c.o
ld: symbol(s) not found for architecture x86_64
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [CMakeFiles/lightfield_bitstream_parsing.dir/build.make:161: examples/lightfield_bitstream_parsing] Error 1
make[1]: *** [CMakeFiles/Makefile2:432: CMakeFiles/lightfield_bitstream_parsing.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 73%] Built target set_maps
[ 73%] Built target scalable_encoder
[ 74%] Building CXX object CMakeFiles/test_aom_common.dir/test/decode_test_driver.cc.o
[ 74%] Building CXX object CMakeFiles/test_aom_common.dir/test/test_vectors.cc.o
[ 74%] Built target test_aom_common
make: *** [Makefile:130: all] Error 2
builder for ‘/nix/store/szw9i81ss8a9a5fkmql8hf8cvyfhxlii-libaom-1.0.0.errata.1.drv’ failed with exit code 2
cannot build derivation ‘/nix/store/z2h2m1fv4fdd9vyn2x3j8yq28159kz38-ffmpeg-full-4.1.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/z2h2m1fv4fdd9vyn2x3j8yq28159kz38-ffmpeg-full-4.1.drv’ failed

Presently this means static, instead of the shared we started building after a recent change (my blame). Darwin build breaks with this, and it was introduced on a whim so instead of making this platform-specific, go back to how things were previously.

infinisil · 2018-12-30T16:17:58Z

(still the same error when building)

dtzWill · 2018-12-30T18:06:17Z

Thank you for testing this and for the feedback! What's going on? Hmm..

Did aom build previously? Yes, it appears so: https://hydra.nixos.org/job/nixpkgs/trunk/libaom.x86_64-darwin (for example: https://hydra.nixos.org/build/86083792/log).

If I'm understanding this right, Borg did try building latest on Darwin... and libaom built successfully, with the failure occurring during the ffmpeg build:

[...]
LD      libavdevice/libavdevice.58.dylib
Undefined symbols for architecture x86_64:
  "_OBJC_CLASS_$_NSDictionary", referenced from:
      objc-class-ref in avfoundation.o
  "_OBJC_EHTYPE_$_NSException", referenced from:
      GCC_except_table4 in avfoundation.o
ld: symbol(s) not found for architecture x86_64
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [ffbuild/library.mak:103: libavdevice/libavdevice.58.dylib] Error 1
builder for '/nix/store/4633kfnwrbqfb6j0xd5hnj1m40g4kjm0-ffmpeg-full-4.1.drv' failed with exit code 2
error: build of '/nix/store/4633kfnwrbqfb6j0xd5hnj1m40g4kjm0-ffmpeg-full-4.1.drv' failed

And yes indeed ffmpeg_4 currently ("previously") builds on Darwin: https://hydra.nixos.org/job/nixpkgs/trunk/ffmpeg_4.x86_64-darwin (example: https://hydra.nixos.org/build/86416146). This is expected but just checking :).

@infinisil are you seeing the same behavior/error as Borg? And if it's not too much trouble: was the last commit (d349cfe) necessary to fix libaom for you (if indeed it now builds again)?

infinisil · 2018-12-30T19:42:39Z

Ah yes, seeing the same as ofborg, here's a reference of the build logs:

On 8ac99ab, libaom failing:

[ 80%] Building ASM object /tmp/nix-build-libaom-1.0.0.drv-0/aom/build/asm_objects/aom_dsp_common_ssse3/aom_subpixel_bilinear_ssse3.asm.o
[ 81%] Building ASM object /tmp/nix-build-libaom-1.0.0.drv-0/aom/build/asm_objects/aom_dsp_encoder_ssse3/fwd_txfm_ssse3_x86_64.asm.o
[ 81%] Building ASM object /tmp/nix-build-libaom-1.0.0.drv-0/aom/build/asm_objects/aom_dsp_encoder_ssse3/ssim_opt_x86_64.asm.o
[ 81%] Building ASM object /tmp/nix-build-libaom-1.0.0.drv-0/aom/build/asm_objects/aom_dsp_encoder_ssse3/quantize_ssse3_x86_64.asm.o
[ 81%] Building ASM object /tmp/nix-build-libaom-1.0.0.drv-0/aom/build/asm_objects/aom_dsp_encoder_avx/quantize_avx_x86_64.asm.o
[ 82%] Building ASM object /tmp/nix-build-libaom-1.0.0.drv-0/aom/build/asm_objects/aom_ports/emms.asm.o
[ 82%] Building ASM object /tmp/nix-build-libaom-1.0.0.drv-0/aom/build/asm_objects/aom_av1_encoder_sse2/dct_sse2.asm.o
[ 82%] Building ASM object /tmp/nix-build-libaom-1.0.0.drv-0/aom/build/asm_objects/aom_av1_encoder_sse2/error_sse2.asm.o
[ 82%] Building ASM object /tmp/nix-build-libaom-1.0.0.drv-0/aom/build/asm_objects/aom_av1_encoder_sse2/temporal_filter_apply_sse2.asm.o
Scanning dependencies of target aom
[ 82%] Building C object CMakeFiles/aom.dir/aom/src/aom_codec.c.o
[ 82%] Building C object CMakeFiles/aom.dir/config/aom_config.c.o
[ 82%] Building C object CMakeFiles/aom.dir/aom/src/aom_decoder.c.o
[ 83%] Building C object CMakeFiles/aom.dir/aom/src/aom_encoder.c.o
[ 83%] Building C object CMakeFiles/aom.dir/aom/src/aom_image.c.o
[ 83%] Building C object CMakeFiles/aom.dir/aom/src/aom_integer.c.o
[ 83%] Linking C shared library libaom.dylib
[ 83%] Built target aom
Scanning dependencies of target set_maps
Scanning dependencies of target lightfield_decoder
Scanning dependencies of target scalable_encoder
[ 83%] Building C object CMakeFiles/set_maps.dir/examples/set_maps.c.o
[ 83%] Building C object CMakeFiles/scalable_encoder.dir/examples/scalable_encoder.c.o
[ 83%] Building C object CMakeFiles/lightfield_decoder.dir/examples/lightfield_decoder.c.o
[ 84%] Linking CXX executable examples/scalable_encoder
[ 84%] Linking CXX executable examples/lightfield_decoder
[ 84%] Linking CXX executable examples/set_maps
Undefined symbols for architecture x86_64:
  "_aom_read_obu_header", referenced from:
      _obudec_read_obu_header_and_size in obudec.c.o
ld: symbol(s) not found for architecture x86_64
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [CMakeFiles/lightfield_decoder.dir/build.make:147: examples/lightfield_decoder] Error 1
make[1]: *** [CMakeFiles/Makefile2:6180: CMakeFiles/lightfield_decoder.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 84%] Built target scalable_encoder
[ 84%] Built target set_maps
make: *** [Makefile:130: all] Error 2
builder for '/nix/store/9s8fdm2b051j6kxx1z0ki2a5r98zn07c-libaom-1.0.0.drv' failed with exit code 2
cannot build derivation '/nix/store/9q1cd26q3kz6igknmrlr97z53dyzx1gh-ffmpeg-full-4.1.drv': 1 dependencies couldn't be built
error: build of '/nix/store/9q1cd26q3kz6igknmrlr97z53dyzx1gh-ffmpeg-full-4.1.drv' failed

On d9c1dd2, libaom failing:

[ 72%] Linking C shared library libaom.dylib
[ 72%] Built target aom
Scanning dependencies of target lightfield_bitstream_parsing
Scanning dependencies of target set_maps
Scanning dependencies of target scalable_encoder
Scanning dependencies of target test_aom_common
[ 72%] Building C object CMakeFiles/set_maps.dir/examples/set_maps.c.o
[ 72%] Building C object CMakeFiles/scalable_encoder.dir/examples/scalable_encoder.c.o
[ 72%] Building C object CMakeFiles/lightfield_bitstream_parsing.dir/examples/lightfield_bitstream_parsing.c.o
[ 72%] Building CXX object CMakeFiles/test_aom_common.dir/test/aom_integer_test.cc.o
/tmp/nix-build-libaom-1.0.0.errata.1.drv-0/aom/examples/lightfield_bitstream_parsing.c
:
393:31: warning
: implicit conversion loses integer precision: 'long long' to 'int' [-Wshorten-64-to-32]
  int num_tile_lists = tl_pts - pts;

      ~~~~~~~~~~~~~~   ~~~~~~~^~~~~

1
 warning
 generated.
[ 72%] Linking CXX executable examples/set_maps
[ 73%] Linking CXX executable examples/scalable_encoder
[ 73%] Linking CXX executable examples/lightfield_bitstream_parsing
Undefined symbols for architecture x86_64:
  "_aom_read_obu_header", referenced from:
      _obudec_read_obu_header_and_size in obudec.c.o
  "_aom_wb_write_literal", referenced from:
      _process_tile_list in lightfield_bitstream_parsing.c.o
  "_aom_wb_write_unsigned_literal", referenced from:
      _process_tile_list in lightfield_bitstream_parsing.c.o
ld: symbol(s) not found for architecture x86_64
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [CMakeFiles/lightfield_bitstream_parsing.dir/build.make:161: examples/lightfield_bitstream_parsing] Error 1
make[1]: *** [CMakeFiles/Makefile2:432: CMakeFiles/lightfield_bitstream_parsing.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[ 73%] Built target scalable_encoder
[ 73%] Built target set_maps
[ 74%] Building CXX object CMakeFiles/test_aom_common.dir/test/decode_test_driver.cc.o
[ 74%] Building CXX object CMakeFiles/test_aom_common.dir/test/test_vectors.cc.o
[ 74%] Built target test_aom_common
make: *** [Makefile:130: all] Error 2
builder for ‘/nix/store/szw9i81ss8a9a5fkmql8hf8cvyfhxlii-libaom-1.0.0.errata.1.drv’ failed with exit code 2
cannot build derivation ‘/nix/store/z2h2m1fv4fdd9vyn2x3j8yq28159kz38-ffmpeg-full-4.1.drv’: 1 dependencies couldn't be built
error: build of ‘/nix/store/z2h2m1fv4fdd9vyn2x3j8yq28159kz38-ffmpeg-full-4.1.drv’ failed

On d349cfe, libaom succeeding, ffmpeg-full failing:

skipping strip -x libswresample/x86/rematrix.o
CC      libswresample/x86/rematrix_init.o
X86ASM  libswresample/x86/resample.o
STRIP   libswresample/x86/resample.o
skipping strip -x libswresample/x86/resample.o
CC      libswresample/x86/resample_init.o
GEN     libswresample/libswresample.ver
LD      libswresample/libswresample.3.dylib
LD      libavcodec/libavcodec.58.dylib
LD      libavformat/libavformat.58.dylib
CC      libavresample/audio_convert.o
CC      libavresample/audio_data.o
CC      libavresample/audio_mix.o
CC      libavresample/audio_mix_matrix.o
CC      libavresample/dither.o
CC      libavresample/options.o
CC      libavresample/resample.o
CC      libavresample/utils.o
X86ASM  libavresample/x86/audio_convert.o
STRIP   libavresample/x86/audio_convert.o
skipping strip -x libavresample/x86/audio_convert.o
CC      libavresample/x86/audio_convert_init.o
X86ASM  libavresample/x86/audio_mix.o
STRIP   libavresample/x86/audio_mix.o
skipping strip -x libavresample/x86/audio_mix.o
CC      libavresample/x86/audio_mix_init.o
X86ASM  libavresample/x86/dither.o
STRIP   libavresample/x86/dither.o
skipping strip -x libavresample/x86/dither.o
CC      libavresample/x86/dither_init.o
GEN     libavresample/libavresample.ver
LD      libavresample/libavresample.4.dylib
LD      libavfilter/libavfilter.7.dylib
LD      libavdevice/libavdevice.58.dylib
Undefined symbols for architecture x86_64:
  "_OBJC_CLASS_$_NSDictionary", referenced from:
      objc-class-ref in avfoundation.o
  "_OBJC_EHTYPE_$_NSException", referenced from:
      GCC_except_table4 in avfoundation.o
ld: symbol(s) not found for architecture x86_64
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [ffbuild/library.mak:103: libavdevice/libavdevice.58.dylib] Error 1
builder for ‘/nix/store/4633kfnwrbqfb6j0xd5hnj1m40g4kjm0-ffmpeg-full-4.1.drv’ failed with exit code 2
error: build of ‘/nix/store/4633kfnwrbqfb6j0xd5hnj1m40g4kjm0-ffmpeg-full-4.1.drv’ failed

dtzWill · 2018-12-31T16:51:33Z

Darwin failure looks awfully similar to this:
#51027 (comment)

Let's see if the same fix works here? (commit coming soon)

dtzWill · 2018-12-31T17:25:22Z

\o/ :confetti_ball: \o/

Look good?

infinisil · 2019-01-01T08:44:02Z

Well ofborg still says it failed, or am I misinterpreting something

(having borg integrated into the checks is somehow really confusing)

vcunat · 2019-01-07T16:42:49Z

Yes, I believe borg did rebuild it on the last commit and failed on Darwin.

MP2E · 2019-01-19T15:14:15Z

Hey! I've been using this branch for over a week with no issues, but I can't seem to find any failing build logs for Darwin. Could I get a link? I would like to help get this merged, since the version of ffmpeg 4.x currently packaged in nixpkgs master fails to build for me..

vcunat · 2019-01-19T15:42:59Z

@MP2E: see the "checks" tab.

vcunat

The only problem I see is ffmpeg-full on darwin, but that seems failing even worse on master, so let's not block on that now. I re-verified builds on the two linux platforms locally.

GrahamcOfBorg added 10.rebuild-darwin: 11-100 10.rebuild-linux: 101-500 labels Dec 17, 2018

dtzWill force-pushed the update/ffmpeg-2018-12 branch from f8c6a45 to 7d6893e Compare December 19, 2018 14:11

dtzWill added 5 commits December 28, 2018 13:06

ffmpeg: 4.0.2 -> 4.0.3

d15818f

https://www.ffmpeg.org/download.html#releases https://git.ffmpeg.org/gitweb/ffmpeg.git/shortlog/n4.0.3

ffmpeg-full: 4.0.2 -> 4.0.3

842e918

ffmpeg_3_4: 3.4.4 -> 3.4.5

6ba05c7

ffmpeg{_4,-full}: 4.0.3 -> 4.1

2420e96

ffmpeg-full: added missing libXext dep, needed for 'xlib' support

8ac99ab

dtzWill force-pushed the update/ffmpeg-2018-12 branch from 7d6893e to 8ac99ab Compare December 28, 2018 19:06

libaom: 1.0.0 -> 1.0.0.errata.1

d9c1dd2

ffmpeg-full: use cf-private to fix Darwin build

4718047

vcunat approved these changes Jan 19, 2019

View reviewed changes

vcunat merged commit 4718047 into NixOS:master Jan 19, 2019

vcunat added a commit that referenced this pull request Jan 19, 2019

Merge #52444: ffmpeg{_3_4,_4,-full}: updates

b277551

orivej mentioned this pull request May 23, 2020

libaom: 1.0.0-errata1 -> 2.0.0 #88439

Merged

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ffmpeg{_3_4,_4,-full}: updates #52444

ffmpeg{_3_4,_4,-full}: updates #52444

dtzWill commented Dec 17, 2018

dtzWill commented Dec 19, 2018

ivan commented Dec 19, 2018

dtzWill commented Dec 19, 2018

ivan commented Dec 19, 2018

dtzWill commented Dec 19, 2018

dtzWill commented Dec 28, 2018

dtzWill commented Dec 29, 2018

infinisil commented Dec 29, 2018

infinisil commented Dec 29, 2018 •

edited

dtzWill commented Dec 30, 2018

dtzWill commented Dec 30, 2018

dtzWill commented Dec 30, 2018

dtzWill commented Dec 30, 2018

infinisil commented Dec 30, 2018

infinisil commented Dec 30, 2018

dtzWill commented Dec 30, 2018

infinisil commented Dec 30, 2018

dtzWill commented Dec 31, 2018

dtzWill commented Dec 31, 2018

infinisil commented Jan 1, 2019 •

edited

vcunat commented Jan 7, 2019

MP2E commented Jan 19, 2019

vcunat commented Jan 19, 2019

vcunat left a comment

ffmpeg{_3_4,_4,-full}: updates #52444

ffmpeg{_3_4,_4,-full}: updates #52444

Conversation

dtzWill commented Dec 17, 2018

Motivation for this change

Things done

dtzWill commented Dec 19, 2018

ivan commented Dec 19, 2018

dtzWill commented Dec 19, 2018

ivan commented Dec 19, 2018

dtzWill commented Dec 19, 2018

dtzWill commented Dec 28, 2018

dtzWill commented Dec 29, 2018

infinisil commented Dec 29, 2018

infinisil commented Dec 29, 2018 • edited

dtzWill commented Dec 30, 2018

dtzWill commented Dec 30, 2018

dtzWill commented Dec 30, 2018

dtzWill commented Dec 30, 2018

infinisil commented Dec 30, 2018

infinisil commented Dec 30, 2018

dtzWill commented Dec 30, 2018

infinisil commented Dec 30, 2018

dtzWill commented Dec 31, 2018

dtzWill commented Dec 31, 2018

infinisil commented Jan 1, 2019 • edited

vcunat commented Jan 7, 2019

MP2E commented Jan 19, 2019

vcunat commented Jan 19, 2019

vcunat left a comment

Choose a reason for hiding this comment

infinisil commented Dec 29, 2018 •

edited

infinisil commented Jan 1, 2019 •

edited