Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

osquery: init at 2.5.2 #27347

Merged
merged 5 commits into from Jul 25, 2017
Merged

osquery: init at 2.5.2 #27347

merged 5 commits into from Jul 25, 2017

Conversation

cstrahan
Copy link
Contributor

@cstrahan cstrahan commented Jul 13, 2017

Motivation for this change

I'd like to use osquery: https://osquery.io/

Things done
  • Tested using sandboxing
    (nix.useSandbox on NixOS,
    or option build-use-sandbox in nix.conf
    on non-NixOS)
  • Built on platform(s)
    • NixOS
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

The osqueryd daemon can be configured in NixOS like so:

{ config, lib, pkgs, ... }:

with lib;

{ services.osquery.enable = true;
  services.osquery.extraConfig = {
    schedule = {
      # a simple example of logging added/removed processes (by pid and name):
      processes = {
        query = "SELECT pid, name FROM processes;";
        interval = 10;
      };
    };
    packs = {
      # standard packs can be referenced like so:
      monitoring = "${pkgs.osquery}/share/osquery/packs/osquery-monitoring.conf";
    };
  };

}

As time goes by, we can try to expand the support for legitimate configuration options, but that's out of scope for this PR; for now, extraConfig provides an escape hatch (we'll want this anyway, since the config file schema is large, and we'll undoubtedly provide only subset as direct NixOS options at any given moment; we should allow people to easily express the config settings in the interim). The settings in extraConfig are merged recursively, so separate NixOS config files can specify separate schedules, packs, etc.

@cstrahan cstrahan changed the title Osquery new osquery: init at 2.5.2 Jul 13, 2017
@cstrahan
Copy link
Contributor Author

This currently builds the SDK as a static library, while it would be nice to instead provide a shared library.

Unfortunately, when I add:

  ${if (!enableStatic) then "OSQUERY_BUILD_SHARED" else null} = true;
  ${if (!enableStatic) then "BUILD_LINK_SHARED" else null} = true;

I get the following compile error:

[ 29%] Linking CXX shared library libosquery.so
cd /tmp/nix-build-osquery-2.5.2.drv-0/osquery-2.5.2-src/build/osquery && /nix/store/5dh82kd6qppsap7b12llcbp6w4d2i6a9-cmake-3.8.2/bin/cmake -E cmake_link_script CMakeFiles/libosquery_shared.dir/link.txt --verbose=1
/nix/store/3k978nnp7ppjim34di40vwz5f95rsxhw-gcc-wrapper-5.4.0/bin/g++ -fPIC -O3 -DNDEBUG  -shared -Wl,-soname,libosquery.so -o libosquery.so CMakeFiles/libosquery_shared.dir/main/lib.cpp.o config/CMakeFiles/osquery_config.dir/config.cpp.o config/CMakeFiles/osquery_config.dir/packs.cpp.o config/CMakeFiles/osquery_config_parsers.dir/parsers/decorators.cpp.o config/CMakeFiles/osquery_config_parsers.dir/parsers/events_parser.cpp.o config/CMakeFiles/osquery_config_parsers.dir/parsers/file_paths.cpp.o config/CMakeFiles/osquery_config_parsers.dir/parsers/options.cpp.o config/CMakeFiles/osquery_config_parsers.dir/parsers/prometheus_targets.cpp.o config/CMakeFiles/osquery_config_parsers.dir/parsers/views.cpp.o core/CMakeFiles/osquery_core.dir/conversions.cpp.o core/CMakeFiles/osquery_core.dir/init.cpp.o core/CMakeFiles/osquery_core.dir/system.cpp.o core/CMakeFiles/osquery_core.dir/posix/process.cpp.o core/CMakeFiles/osquery_core.dir/posix/process_ops.cpp.o core/CMakeFiles/osquery_core.dir/posix/utils.cpp.o core/CMakeFiles/osquery_core.dir/tables.cpp.o core/CMakeFiles/osquery_core.dir/flags.cpp.o core/CMakeFiles/osquery_core.dir/watcher.cpp.o database/CMakeFiles/osquery_database.dir/database.cpp.o database/CMakeFiles/osquery_database.dir/query.cpp.o database/CMakeFiles/osquery_database.dir/plugins/ephemeral.cpp.o dispatcher/CMakeFiles/osquery_dispatcher.dir/dispatcher.cpp.o distributed/CMakeFiles/osquery_distributed.dir/distributed.cpp.o events/CMakeFiles/osquery_events.dir/events.cpp.o extensions/CMakeFiles/osquery_extensions.dir/__/__/generated/gen-cpp/Extension.cpp.o extensions/CMakeFiles/osquery_extensions.dir/__/__/generated/gen-cpp/ExtensionManager.cpp.o extensions/CMakeFiles/osquery_extensions.dir/__/__/generated/gen-cpp/osquery_types.cpp.o extensions/CMakeFiles/osquery_extensions.dir/extensions.cpp.o extensions/CMakeFiles/osquery_extensions.dir/interface.cpp.o filesystem/CMakeFiles/osquery_filesystem_linux.dir/linux/mem.cpp.o filesystem/CMakeFiles/osquery_filesystem_linux.dir/linux/proc.cpp.o filesystem/CMakeFiles/osquery_filesystem.dir/filesystem.cpp.o filesystem/CMakeFiles/osquery_filesystem.dir/posix/fileops.cpp.o logger/CMakeFiles/osquery_logger.dir/logger.cpp.o registry/CMakeFiles/osquery_registry.dir/registry.cpp.o sql/CMakeFiles/osquery_sql.dir/sql.cpp.o tables/CMakeFiles/osquery_tables_utility.dir/utility/file.cpp.o tables/CMakeFiles/osquery_tables_utility.dir/utility/osquery.cpp.o tables/CMakeFiles/osquery_tables_utility.dir/utility/time.cpp.o CMakeFiles/osquery_amalgamation.dir/__/generated/utils_amalgamation.cpp.o -lpthread -lz -lgflags -lthrift -llz4 -ldl -lsqlite3 -lboost_system -lboost_filesystem -lboost_thread -lboost_context -lrt -lc -lglog -llzma -lbz2 -rdynamic -Wl,-zrelro -Wl,-znow -pie -lgcc_s -Wl,--build-id
/nix/store/f111ij1fc83965m48bf2zqgiaq88fqv5-glibc-2.25/lib/Scrt1.o: In function `_start':
/build/glibc-2.25/csu/../sysdeps/x86_64/start.S:104: undefined reference to `main'
collect2: error: ld returned 1 exit status

wantedBy = [ "multi-user.target" ];
path = [ pkgs.osquery ];
preStart = ''
mkdir -p ${escapeShellArg cfg.loggerPath}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

escapeShellArg will not add quotes around the argument. You have to add them manually.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not what it looks like:

nix-repl> lib.escapeShellArg "this is a test"
"'this is a test'"

Note, specifically, the inner single quotes (').

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right.

GENTOO_RELEASE = "/etc/gentoo-release"

def _platform():
+ return ("nixos", "nixos")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To bad that they have no support for /etc/os-release. The only disadvantage of this patch is, that it won't work on non-nixos. Also I don't how many people would use osqueryd that way.

else()
set(LINUX TRUE)
- set(CXX_COMPILE_FLAGS "${CXX_COMPILE_FLAGS} -std=c++14 -stdlib=libstdc++")
+ set(CXX_COMPILE_FLAGS "${CXX_COMPILE_FLAGS} -std=c++14")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the problem with libstdc++ in nixos? It might also helpful to add a comment so future maintainer have an easier job.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Mic92 I get: g++: error: unrecognized command line option '-stdlib=libstdc++'

The osquery devs build their stuff with clang, where that flag makes sense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do think we could run into subtle runtime bugs due not using clang++ on nixos?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No; we're not mixing different standard libs, so everything should be fine.

@cstrahan
Copy link
Contributor Author

I've further split-up the commits (factoring out rocksdb-lite and linenoise-ng).

@cstrahan
Copy link
Contributor Author

I think this is fairly uncontroversial; merging.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants