Upgrade InfluxDB storage to InfluxDB 0.9 #1040

jimmidyson · 2016-01-07T09:31:11Z

Supersedes #800 by adding missing godeps.

/cc @marcellodesales @christianhuening @carmark @mnuessler

googlebot · 2016-01-07T09:31:14Z

We found a Contributor License Agreement for you (the sender of this pull request), but were unable to find agreements for the commit author(s). If you authored these, maybe you used a different email address in the git commits than was used to sign the CLA (login here to double check)? If these were authored by someone else, then they will need to sign a CLA as well, and confirm that they're okay with these being contributed to Google.

k8s-bot · 2016-01-07T09:34:02Z

Jenkins GCE e2e

Build/test failed for commit 99cb43b.

Build Log

jimmidyson · 2016-01-07T09:36:09Z

Unrelated flakes... Travis failing because of go vet changes it seems & e2e failing with:

ERROR: (gcloud.compute.ssh) Your current active account [211744435552-compute@developer.gserviceaccount.com] does not have any valid credentials\nPlease run:\n\n  $ gcloud auth login\n\nto obtain new credentials, or if you have already logged in with a\ndifferent account:\n\n  $ gcloud config set account ACCOUNT\n\nto select an already authenticated account to use.

googlebot · 2016-01-07T10:21:58Z

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

k8s-bot · 2016-01-07T10:23:40Z

Jenkins GCE e2e

Build/test failed for commit 46eded2.

Build Log

k8s-bot · 2016-01-07T10:25:36Z

Jenkins GCE e2e

Build/test passed for commit b903b75.

Build Log

carmark · 2016-01-07T14:25:53Z

Shall we refactor the function of StatsEq with reflect.DeepEqual in storage/influxdb/influxdb_test.go?

jimmidyson · 2016-01-07T14:30:29Z

@carmark Unless you feel strongly about it I'd leave it as is.

- Fix google#743 - Rewrite InfluxDB storage for new InfluxDB API data structures. - Store each measurement separately instead of storing all measurements in a single big "table" with many columns/fields. - Use tags add metadata to points, such as the container name. Tags are a new feature in InfluxDB 0.9.

k8s-bot · 2016-01-07T14:35:42Z

Jenkins GCE e2e

Build/test passed for commit 49ee94d.

Build Log

jimmidyson · 2016-01-07T14:41:49Z

This passing tests now - would like to get this merged ASAP

jimmidyson · 2016-01-07T14:47:21Z

/cc @vishh

normanjoyner · 2016-01-07T15:59:04Z

👍

vishh · 2016-01-07T17:14:09Z

storage/influxdb/influxdb.go

-	colContainerName      string = "container_name"
-	colCpuCumulativeUsage string = "cpu_cumulative_usage"
+	// Cumulative CPU usage
+	serCpuUsageTotal  string = "cpu_usage_total"


Can we use - instead? Also why drop the cumulative keyword?

Using - may mean users have to quote everything in their queries, otherwise might be seen as operator.

Cumulative dropped to be consistent with other serCpuUsage* vars below.

Ok. May be we can document the schema instead of including the cumulative keyword.

vishh · 2016-01-07T17:35:03Z

IN general LGTM. We can fix those comments later.

k8s-bot · 2016-01-07T17:36:38Z

Jenkins GCE e2e

Build/test passed for commit 05b435e.

Build Log

k8s-bot · 2016-01-07T17:39:40Z

Jenkins GCE e2e

Build/test passed for commit 706a954.

Build Log

marcellodesales · 2016-01-07T18:31:19Z

@jimmidyson I hope I get some time to build local and verify in case this tests pass...

carmark · 2016-01-08T08:23:23Z

@jimmidyson Fine for me, may modify them later.

jimmidyson · 2016-01-08T08:53:30Z

@marcellodesales Tests pass so please try locally when you can.

@vishh Happy to merge then? We can clean up anything afterwards if need be but all LGTM.

Upgrade InfluxDB storage to InfluxDB 0.9

marcellodesales · 2016-01-09T02:57:57Z

@jimmidyson I will try to build and verify this weekend! Thanks a lot for merging this!

svenmueller · 2016-01-26T23:25:39Z

which version does contain this commit?

marcellodesales · 2016-01-28T06:47:25Z

@svenmueller apparently not released yet...

Sorry... took me a while to try to verify this...

@jimmidyson, I apparently can't get it to work yet... I just built an image at https://hub.docker.com/r/marcellodesales/google-cadvisor/, tag influxdb-0.9, based on commit 040bdd3 (see step-by-step)...

UPDATE: IT WORKS! -storage_driver=influxdb was missing in the command instruction below.
UPDATE: Docker-compose at #1040 (comment)

How I built

Could you please verify if that's correct? I did the following (https://hub.docker.com/r/marcellodesales/google-cadvisor/)

Checked out commit 040bdd3
Built locally
Ran local tests
Built Docker Image
Tagged and Pushed

Build Instructions

Got the code

https://github.com/google/cadvisor/blob/master/docs/build.md

$ go get -d github.com/google/cadvisor
package golang.org/x/net/context/ctxhttp: /home/mdesales/go/src/golang.org/x/net exists but /home/mdesales/go/src/golang.org/x/net/.git does not - stale checkout?
y
$ go get github.com/tools/godep

Checked out the merge commit

 $ git checkout 040bdd3c
Note: checking out '040bdd3c'.

$ git log
commit 040bdd3cb1b038f2b6aca48c62fa4a1a8cb8da6c
Merge: 1b62cef 706a954
Author: Vish Kannan <vishh@users.noreply.github.com>
Date:   Fri Jan 8 10:27:33 2016 -0800

    Merge pull request #1040 from jimmidyson/influxdb-0.9

    Upgrade InfluxDB storage to InfluxDB 0.9

Built, verified

$ godep go build .

$ ls -la cadvisor
-rwxr-xr-x 1 mdesales mdesales 19507383 Jan 27 16:15 cadvisor

$ file cadvisor
cadvisor: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.24, BuildID[sha1]=befe7321680d19977d844d01e08f67af328eabe8, not stripped

Ran Unit test cases

$ godep go test ./... -test.short
?       github.com/google/cadvisor  [no test files]
ok      github.com/google/cadvisor/api  0.018s
?       github.com/google/cadvisor/cache    [no test files]
ok      github.com/google/cadvisor/cache/memory 0.324s
ok      github.com/google/cadvisor/client   0.685s
?       github.com/google/cadvisor/client/clientexample [no test files]
ok      github.com/google/cadvisor/client/v2    0.042s
ok      github.com/google/cadvisor/collector    0.043s
ok      github.com/google/cadvisor/container    0.010s
?       github.com/google/cadvisor/container/docker [no test files]
ok      github.com/google/cadvisor/container/libcontainer   0.044s
ok      github.com/google/cadvisor/container/raw    0.030s
ok      github.com/google/cadvisor/events   0.011s
ok      github.com/google/cadvisor/fs   0.740s
?       github.com/google/cadvisor/healthz  [no test files]
?       github.com/google/cadvisor/http [no test files]
?       github.com/google/cadvisor/http/mux [no test files]
ok      github.com/google/cadvisor/info/v1  0.072s
?       github.com/google/cadvisor/info/v1/test [no test files]
?       github.com/google/cadvisor/info/v2  [no test files]
?       github.com/google/cadvisor/integration/common   [no test files]
?       github.com/google/cadvisor/integration/framework    [no test files]
?       github.com/google/cadvisor/integration/runner   [no test files]
ok      github.com/google/cadvisor/integration/tests/api    0.023s
ok      github.com/google/cadvisor/integration/tests/healthz    0.003s
ok      github.com/google/cadvisor/manager  0.036s
ok      github.com/google/cadvisor/metrics  0.007s
?       github.com/google/cadvisor/pages    [no test files]
?       github.com/google/cadvisor/pages/static [no test files]
?       github.com/google/cadvisor/storage  [no test files]
?       github.com/google/cadvisor/storage/bigquery [no test files]
?       github.com/google/cadvisor/storage/bigquery/client  [no test files]
?       github.com/google/cadvisor/storage/bigquery/client/example  [no test files]
?       github.com/google/cadvisor/storage/elasticsearch    [no test files]
?       github.com/google/cadvisor/storage/influxdb [no test files]
?       github.com/google/cadvisor/storage/redis    [no test files]
?       github.com/google/cadvisor/storage/statsd   [no test files]
?       github.com/google/cadvisor/storage/statsd/client    [no test files]
?       github.com/google/cadvisor/storage/stdout   [no test files]
?       github.com/google/cadvisor/storage/test [no test files]
ok      github.com/google/cadvisor/summary  0.026s
ok      github.com/google/cadvisor/utils    0.003s
?       github.com/google/cadvisor/utils/cloudinfo  [no test files]
?       github.com/google/cadvisor/utils/cpuload    [no test files]
?       github.com/google/cadvisor/utils/cpuload/netlink    [no test files]
?       github.com/google/cadvisor/utils/cpuload/netlink/example    [no test files]
ok      github.com/google/cadvisor/utils/machine    0.005s
ok      github.com/google/cadvisor/utils/oomparser  0.005s
?       github.com/google/cadvisor/utils/oomparser/oomexample   [no test files]
?       github.com/google/cadvisor/utils/procfs [no test files]
?       github.com/google/cadvisor/utils/sysfs  [no test files]
?       github.com/google/cadvisor/utils/sysfs/fakesysfs    [no test files]
ok      github.com/google/cadvisor/utils/sysinfo    0.002s
ok      github.com/google/cadvisor/validate 0.008s
?       github.com/google/cadvisor/version  [no test files]

Generated the new Docker Image to push to https://hub.docker.com/r/marcellodesales/google-cadvisor/.

$ make docker
Sending build context to Docker daemon 46.42 MB
Step 1 : FROM alpine:3.2
 ---> ab7e84202862
Step 2 : MAINTAINER dengnan@google.com vmarmol@google.com vishnuk@google.com jimmidyson@gmail.com
 ---> Using cache
 ---> 88671c112252
Step 3 : RUN apk add --update ca-certificates device-mapper &&     wget https://circle-artifacts.com/gh/andyshinn/alpine-pkg-glibc/8/artifacts/0/home/ubuntu/alpine-pkg-glibc/packages/x86_64/glibc-2.21-r2.apk &&     wget https://circle-artifacts.com/gh/andyshinn/alpine-pkg-glibc/8/artifacts/0/home/ubuntu/alpine-pkg-glibc/packages/x86_64/glibc-bin-2.21-r2.apk &&     apk add --allow-untrusted glibc-2.21-r2.apk glibc-bin-2.21-r2.apk &&     /usr/glibc/usr/bin/ldconfig /lib /usr/glibc/usr/lib &&     echo 'hosts: files mdns4_minimal [NOTFOUND=return] dns mdns4' >> /etc/nsswitch.conf &&     rm -rf /var/cache/apk/*
 ---> Running in da7cd7d9b138
fetch http://dl-4.alpinelinux.org/alpine/v3.2/main/x86_64/APKINDEX.tar.gz
(1/9) Installing run-parts (4.4-r0)
(2/9) Installing openssl (1.0.2e-r0)
(3/9) Installing lua5.2-libs (5.2.4-r0)
(4/9) Installing lua5.2 (5.2.4-r0)
(5/9) Installing ncurses-terminfo-base (5.9-r3)
(6/9) Installing ncurses-widec-libs (5.9-r3)
(7/9) Installing lua5.2-posix (33.3.1-r2)
(8/9) Installing ca-certificates (20141019-r2)
(9/9) Installing device-mapper (2.02.122-r3)
Executing busybox-1.23.2-r0.trigger
Executing ca-certificates-20141019-r2.trigger
OK: 8 MiB in 24 packages
Connecting to circle-artifacts.com (52.72.7.255:443)
glibc-2.21-r2.apk    100% |*******************************|  1813k  0:00:00 ETA
Connecting to circle-artifacts.com (52.71.104.154:443)
glibc-bin-2.21-r2.ap 100% |*******************************|  1018k  0:00:00 ETA
(1/3) Installing libgcc (4.9.2-r5)
(2/3) Installing glibc (2.21-r2)
(3/3) Installing glibc-bin (2.21-r2)
Executing glibc-bin-2.21-r2.trigger
OK: 15 MiB in 27 packages
 ---> dd061e106fba
Removing intermediate container da7cd7d9b138
Step 4 : ADD cadvisor /usr/bin/cadvisor
 ---> aa66b7ae0141
Removing intermediate container e73877928c84
Step 5 : EXPOSE 8080
 ---> Running in dafae0aa3f2e
 ---> 58e6ab9cc383
Removing intermediate container dafae0aa3f2e
Step 6 : ENTRYPOINT /usr/bin/cadvisor -logtostderr
 ---> Running in 8eff24631902
 ---> b6495090befb
Removing intermediate container 8eff24631902
Successfully built b6495090befb

 $ docker images | grep cadvisor
cadvisor                                                  040bdd3                       b6495090befb        8 seconds ago       36.52 MB

$ docker tag b6495090befb marcellodesales/google-cadvisor:influxdb-0.9

InfluxDB setup

Started a regular latest InfluxDB with default paramers

$ sudo docker run -d -p 8083:8083 -p 8086:8086 --expose 8090 --expose 8099 --name influxsrv tutum/influxdb

I created the default user root:root as it was not created at bootstrap.

I also created the default database cadvisor

$ docker logs influxsrv
influxdb configuration: 
### Welcome to the InfluxDB configuration file.

# Once every 24 hours InfluxDB will report anonymous data to m.influxdb.com
# The data includes raft id (random 8 bytes), os, arch, version, and metadata.
# We don't track ip addresses of servers reporting. This is only used
# to track the number of instances running and the versions, which
# is very helpful for us.
# Change this option to true to disable reporting.
reporting-disabled = false

###
### [meta]
###
### Controls the parameters for the Raft consensus group that stores metadata
### about the InfluxDB cluster.
###

[meta]
  dir = "/data/meta"
  hostname = "localhost"
  bind-address = ":8088"
  retention-autocreate = true
  election-timeout = "1s"
  heartbeat-timeout = "1s"
  leader-lease-timeout = "500ms"
  commit-timeout = "50ms"

###
### [data]
###
### Controls where the actual shard data for InfluxDB lives and how it is
### flushed from the WAL. "dir" may need to be changed to a suitable place
### for your system, but the WAL settings are an advanced configuration. The
### defaults should work for most systems.
###

[data]
  dir = "/data/db"

  # The following WAL settings are for the b1 storage engine used in 0.9.2. They won't
  # apply to any new shards created after upgrading to a version > 0.9.3.
  max-wal-size = 104857600 # Maximum size the WAL can reach before a flush. Defaults to 100MB.
  wal-flush-interval = "10m0s" # Maximum time data can sit in WAL before a flush.
  wal-partition-flush-delay = "2s" # The delay time between each WAL partition being flushed.

  # These are the WAL settings for the storage engine >= 0.9.3
  wal-dir = "/data/wal"
  wal-enable-logging = true

  # When a series in the WAL in-memory cache reaches this size in bytes it is marked as ready to
  # flush to the index
  # wal-ready-series-size = 25600

  # Flush and compact a partition once this ratio of series are over the ready size
  # wal-compaction-threshold = 0.6

  # Force a flush and compaction if any series in a partition gets above this size in bytes
  # wal-max-series-size = 2097152

  # Force a flush of all series and full compaction if there have been no writes in this
  # amount of time. This is useful for ensuring that shards that are cold for writes don't
  # keep a bunch of data cached in memory and in the WAL.
  # wal-flush-cold-interval = "10m"

  # Force a partition to flush its largest series if it reaches this approximate size in
  # bytes. Remember there are 5 partitions so you'll need at least 5x this amount of memory.
  # The more memory you have, the bigger this can be.
  # wal-partition-size-threshold = 20971520
..
..

cAdvisor Logs

So, I tried to run locally and I still cannot see anything in InfluxDB, default parameters...

$ sudo docker run --volume=/:/rootfs:ro --volume=/var/run:/var/run:rw --volume=/sys:/sys:ro --volume=/var/lib/docker/:/var/lib/docker:ro --publish=9080:8080 --detach=true --name=cadvisor-local2 marcellodesales/google-cadvisor:influxdb-0.9 -storage_driver_db=cadvisor -storage_driver_host=localhost:8086

Here's the output... Is there any switch to turn on debugger around the connector?

$ docker logs cadvisor
I0128 01:31:19.446159       1 storagedriver.go:44] Caching stats in memory for 2m0s
I0128 01:31:19.446414       1 manager.go:128] cAdvisor running in container: "/docker/6a522685230ed83cabc1097bb19106353f38821cd99455567655908d74926b3a"
I0128 01:31:19.622739       1 fs.go:105] Filesystem partitions: map[/dev/disk/by-uuid/aef7d545-5fd5-488b-b69a-2b460a8daaef:{mountpoint:/rootfs major:8 minor:1 fsType: blockSize:0} /dev/sda3:{mountpoint:/rootfs/more-space major:8 minor:3 fsType: blockSize:0}]
I0128 01:31:19.754555       1 machine.go:50] Couldn't collect info from any of the files in "/rootfs/etc/machine-id,/var/lib/dbus/machine-id"
I0128 01:31:19.754676       1 manager.go:163] Machine: {NumCores:4 CpuFrequency:2793746 MemoryCapacity:7172173824 MachineID: SystemUUID:564DB8D8-A255-24BE-28A2-370A36553D36 BootID:2dfc9f7d-ece4-479c-921c-ca71e3fa067d Filesystems:[{Device:/dev/disk/by-uuid/aef7d545-5fd5-488b-b69a-2b460a8daaef Capacity:19945680896} {Device:/dev/sda3 Capacity:52710469632}] DiskMap:map[2:0:{Name:fd0 Major:2 Minor:0 Size:0 Scheduler:deadline} 8:0:{Name:sda Major:8 Minor:0 Size:75161927680 Scheduler:deadline} 8:16:{Name:sdb Major:8 Minor:16 Size:107374182400 Scheduler:deadline}] NetworkDevices:[{Name:br-82cc809b176a MacAddress:02:42:30:9f:da:23 Speed:0 Mtu:1500} {Name:br-cb5e14d68103 MacAddress:02:42:af:8b:78:a5 Speed:0 Mtu:1500} {Name:eth0 MacAddress:00:0c:29:55:3d:36 Speed:1000 Mtu:1500}] Topology:[{Id:0 Memory:7172173824 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:6291456 Type:Unified Level:3} {Size:134217728 Type:Unified Level:4}]} {Id:2 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:6291456 Type:Unified Level:3} {Size:134217728 Type:Unified Level:4}]} {Id:4 Memory:0 Cores:[{Id:0 Threads:[2] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:6291456 Type:Unified Level:3} {Size:134217728 Type:Unified Level:4}]} {Id:6 Memory:0 Cores:[{Id:0 Threads:[3] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:6291456 Type:Unified Level:3} {Size:134217728 Type:Unified Level:4}]}] CloudProvider:Unknown InstanceType:Unknown}
I0128 01:31:19.756000       1 manager.go:169] Version: {KernelVersion:3.13.0-76-generic ContainerOsVersion:Alpine Linux v3.2 DockerVersion:1.9.1 CadvisorVersion: CadvisorRevision:}
I0128 01:31:19.883034       1 factory.go:245] Registering Docker factory
I0128 01:31:19.886669       1 factory.go:94] Registering Raw factory
I0128 01:31:20.154208       1 manager.go:1005] Started watching for new ooms in manager
W0128 01:31:20.154409       1 manager.go:236] Could not configure a source for OOM detection, disabling OOM events: exec: "journalctl": executable file not found in $PATH
I0128 01:31:20.174621       1 manager.go:249] Starting recovery of all containers
I0128 01:31:20.192585       1 manager.go:254] Recovery completed
I0128 01:31:20.201919       1 cadvisor.go:106] Starting cAdvisor version: - on port 8080

I do see activity on the cAdvisor container through the UI,

but I don't see any measurements on the influxDB side.

I do see the measurements for the _internal database...

@jimmidyson, @carmark, @vishh, @christianhuening @carmark @mnuessler Is there anything that can help me debug this?

mnuessler · 2016-01-28T09:03:37Z

I think you may be missing the -storage_driver=influxdb option when running cAdvisor.

marcellodesales · 2016-01-28T09:21:51Z

@mnuessler I finally can confirm the PR is working as expected! Thanks a lot for spotting this mistake 👍

@jimmidyson I can finally confirm this is working for me!!! Thanks a lot!

I do se the first line of the logs 1 storagedriver.go:42] Using backend storage type "influxdb". Here's the full logs...

$ docker logs -f cadvisor-0.9
I0128 09:15:32.838890       1 storagedriver.go:42] Using backend storage type "influxdb"
I0128 09:15:32.838962       1 storagedriver.go:44] Caching stats in memory for 2m0s
I0128 09:15:32.839018       1 manager.go:128] cAdvisor running in container: "/docker/4e56068ff373fc4fee042f052055bb3839c8568cb0fc2347fe411c34fc4efb01"
I0128 09:15:32.975282       1 fs.go:105] Filesystem partitions: map[/dev/disk/by-uuid/aef7d545-5fd5-488b-b69a-2b460a8daaef:{mountpoint:/rootfs major:8 minor:1 fsType: blockSize:0} /dev/sda3:{mountpoint:/rootfs/more-space major:8 minor:3 fsType: blockSize:0}]
I0128 09:15:33.733564       1 machine.go:50] Couldn't collect info from any of the files in "/rootfs/etc/machine-id,/var/lib/dbus/machine-id"
I0128 09:15:33.733781       1 manager.go:163] Machine: {NumCores:4 CpuFrequency:2793746 MemoryCapacity:7172173824 MachineID: SystemUUID:564DB8D8-A255-24BE-28A2-370A36553D36 BootID:2dfc9f7d-ece4-479c-921c-ca71e3fa067d Filesystems:[{Device:/dev/sda3 Capacity:52710469632} {Device:/dev/disk/by-uuid/aef7d545-5fd5-488b-b69a-2b460a8daaef Capacity:19945680896}] DiskMap:map[2:0:{Name:fd0 Major:2 Minor:0 Size:0 Scheduler:deadline} 8:0:{Name:sda Major:8 Minor:0 Size:75161927680 Scheduler:deadline} 8:16:{Name:sdb Major:8 Minor:16 Size:107374182400 Scheduler:deadline}] NetworkDevices:[{Name:br-82cc809b176a MacAddress:02:42:30:9f:da:23 Speed:0 Mtu:1500} {Name:br-cb5e14d68103 MacAddress:02:42:af:8b:78:a5 Speed:0 Mtu:1500} {Name:eth0 MacAddress:00:0c:29:55:3d:36 Speed:1000 Mtu:1500}] Topology:[{Id:0 Memory:7172173824 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:6291456 Type:Unified Level:3} {Size:134217728 Type:Unified Level:4}]} {Id:2 Memory:0 Cores:[{Id:0 Threads:[1] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:6291456 Type:Unified Level:3} {Size:134217728 Type:Unified Level:4}]} {Id:4 Memory:0 Cores:[{Id:0 Threads:[2] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:6291456 Type:Unified Level:3} {Size:134217728 Type:Unified Level:4}]} {Id:6 Memory:0 Cores:[{Id:0 Threads:[3] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:6291456 Type:Unified Level:3} {Size:134217728 Type:Unified Level:4}]}] CloudProvider:Unknown InstanceType:Unknown}
I0128 09:15:33.734994       1 manager.go:169] Version: {KernelVersion:3.13.0-76-generic ContainerOsVersion:Alpine Linux v3.2 DockerVersion:1.9.1 CadvisorVersion: CadvisorRevision:}
I0128 09:15:33.875937       1 factory.go:245] Registering Docker factory
I0128 09:15:33.878857       1 factory.go:94] Registering Raw factory
I0128 09:15:34.161698       1 manager.go:1005] Started watching for new ooms in manager
W0128 09:15:34.161878       1 manager.go:236] Could not configure a source for OOM detection, disabling OOM events: exec: "journalctl": executable file not found in $PATH
I0128 09:15:34.173012       1 manager.go:249] Starting recovery of all containers
I0128 09:15:34.190601       1 manager.go:254] Recovery completed
I0128 09:15:34.200440       1 cadvisor.go:106] Starting cAdvisor version: - on port 8080

Now I can see the measuremnts.

jimmidyson · 2016-01-28T09:28:46Z

@mnuessler Good spot!

@marcellodesales Yay! Thanks for trying it out & confirming it all works as expected.

marcellodesales · 2016-02-02T15:17:55Z

@jimmidyson No problem!!! Any plans to release a version with this?

marcellodesales · 2016-02-04T01:43:25Z

Here's a Docker-compose if anyone wants to use it until this is released.

influxdbData:
  image: busybox
  volumes:
    - ./data/influxdb:/data

influxdb:
  image: tutum/influxdb
  environment:
    - PRE_CREATE_DB=cadvisor
  ports:
    - "8083:8083"
    - "8086:8086"
  expose:
    - "8090"
    - "8099"
  volumes_from:
    - "influxdbData"

cadvisor:
  image: marcellodesales/google-cadvisor:influxdb-0.9
  command: -storage_driver=influxdb -storage_driver_db=cadvisor -storage_driver_host=influxsrv:8086
  ports:
    - "9090:8080"
  volumes:
    - /:/rootfs:ro
    - /var/run:/var/run:rw
    - /sys:/sys:ro
    - /var/lib/docker/:/var/lib/docker:ro
  links:
    - influxdb:influxsrv

marcellodesales · 2016-02-05T19:40:16Z

As discussed at #794 (comment), the fix is apparently only applied to tag v0.20.5. So, here's an updated docker-compose.yml with the changes.

influxdbData:
  image: busybox
  volumes:
    - ./data/influxdb:/data

influxdb:
  image: tutum/influxdb
  environment:
    - PRE_CREATE_DB=cadvisor
  ports:
    - "8083:8083"
    - "8086:8086"
  expose:
    - "8090"
    - "8099"
  volumes_from:
    - "influxdbData"

cadvisor:
  image: google/cadvisor:v0.20.5
  command: -storage_driver=influxdb -storage_driver_db=cadvisor -storage_driver_host=influxsrv:8086
  ports:
    - "9090:8080"
  volumes:
    - /:/rootfs:ro
    - /var/run:/var/run:rw
    - /sys:/sys:ro
    - /var/lib/docker/:/var/lib/docker:ro
  links:
    - influxdb:influxsrv

This worked properly and the logs show the events being properly written.

influxdb_1     | [http] 2016/02/05 19:25:47 172.17.0.2 - root [05/Feb/2016:19:25:47 +0000] POST /write?consistency=&db=cadvisor&precision=&rp= HTTP/1.1 204 0 - cAdvisor/0.20.5 41f62cc0-cc3e-11e5-8007-000000000000 24.01661ms

The latest tag still results in the 404 errors.

influxdb_1     | [wal] 2016/02/05 18:41:22 write to index of partition 1 took 1.451106ms
cadvisor_1     | E0205 18:41:26.374037       1 memory.go:91] failed to write stats to influxDb - Server returned (404): 404 page not found
influxdb_1     | [wal] 2016/02/05 18:41:32 Flush due to idle. Flushing 10 series with 10 points and 494 bytes from partition 1

This commit upgrades docker-compose with the latest cAdvisor, compatible with the latest version of InfluxDB. This was part of the google/cadvisor#1040 (comment).

nuxman · 2016-02-19T17:28:30Z

Thanks a Lot @marcellodesales

marcellodesales · 2016-02-21T02:31:50Z

You're very welcome @nuxman!

jimmidyson force-pushed the influxdb-0.9 branch from 99cb43b to 46eded2 Compare January 7, 2016 10:21

jimmidyson force-pushed the influxdb-0.9 branch from 46eded2 to b903b75 Compare January 7, 2016 10:22

mnuessler and others added 3 commits January 7, 2016 14:32

Update InfluxDB dependency to v0.9.5.1

071e910

Fix up missing godeps

49ee94d

jimmidyson force-pushed the influxdb-0.9 branch from b903b75 to 49ee94d Compare January 7, 2016 14:32

jimmidyson mentioned this pull request Jan 7, 2016

Upgrade InfluxDB storage to InfluxDB 0.9 #800

Closed

vishh reviewed Jan 7, 2016
View reviewed changes

Tidy up conversion type switch

05b435e

Rename influxdb field

706a954

vishh added a commit that referenced this pull request Jan 8, 2016

Merge pull request #1040 from jimmidyson/influxdb-0.9

040bdd3

Upgrade InfluxDB storage to InfluxDB 0.9

vishh merged commit 040bdd3 into google:master Jan 8, 2016

marcellodesales mentioned this pull request Jan 28, 2016

Broken with InfluxDB 0.9.x and latest cAdvisor PR (to support 0.9.x) vegasbrianc/docker-monitoring#3

Closed

marcellodesales mentioned this pull request Feb 2, 2016

failed to write stats to influxDb 0.9.x - Server returned (404): 404 page not found #1028

Closed

marcellodesales mentioned this pull request Feb 4, 2016

Cadvisor fails to write stats to influxDB #794

Closed

marcellodesales mentioned this pull request Feb 14, 2016

Update docker-compose.yml with latest cAdvisor and InfluxDB 0.9 vegasbrianc/docker-monitoring#4

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade InfluxDB storage to InfluxDB 0.9 #1040

Upgrade InfluxDB storage to InfluxDB 0.9 #1040

jimmidyson commented Jan 7, 2016

googlebot commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

jimmidyson commented Jan 7, 2016

googlebot commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

carmark commented Jan 7, 2016

jimmidyson commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

jimmidyson commented Jan 7, 2016

jimmidyson commented Jan 7, 2016

normanjoyner commented Jan 7, 2016

vishh Jan 7, 2016

jimmidyson Jan 7, 2016

jimmidyson Jan 7, 2016

vishh Jan 7, 2016

vishh commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

marcellodesales commented Jan 7, 2016

carmark commented Jan 8, 2016

jimmidyson commented Jan 8, 2016

marcellodesales commented Jan 9, 2016

svenmueller commented Jan 26, 2016

marcellodesales commented Jan 28, 2016

mnuessler commented Jan 28, 2016

marcellodesales commented Jan 28, 2016

jimmidyson commented Jan 28, 2016

marcellodesales commented Feb 2, 2016

marcellodesales commented Feb 4, 2016

marcellodesales commented Feb 5, 2016

nuxman commented Feb 19, 2016

marcellodesales commented Feb 21, 2016

Upgrade InfluxDB storage to InfluxDB 0.9 #1040

Upgrade InfluxDB storage to InfluxDB 0.9 #1040

Conversation

jimmidyson commented Jan 7, 2016

googlebot commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

jimmidyson commented Jan 7, 2016

googlebot commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

carmark commented Jan 7, 2016

jimmidyson commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

jimmidyson commented Jan 7, 2016

jimmidyson commented Jan 7, 2016

normanjoyner commented Jan 7, 2016

vishh Jan 7, 2016

Choose a reason for hiding this comment

jimmidyson Jan 7, 2016

Choose a reason for hiding this comment

jimmidyson Jan 7, 2016

Choose a reason for hiding this comment

vishh Jan 7, 2016

Choose a reason for hiding this comment

vishh commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

k8s-bot commented Jan 7, 2016

marcellodesales commented Jan 7, 2016

carmark commented Jan 8, 2016

jimmidyson commented Jan 8, 2016

marcellodesales commented Jan 9, 2016

svenmueller commented Jan 26, 2016

marcellodesales commented Jan 28, 2016

How I built

Build Instructions

Got the code

Checked out the merge commit

Built, verified

Ran Unit test cases

Generated the new Docker Image to push to https://hub.docker.com/r/marcellodesales/google-cadvisor/.

InfluxDB setup

cAdvisor Logs

mnuessler commented Jan 28, 2016

marcellodesales commented Jan 28, 2016

jimmidyson commented Jan 28, 2016

marcellodesales commented Feb 2, 2016

marcellodesales commented Feb 4, 2016

marcellodesales commented Feb 5, 2016

nuxman commented Feb 19, 2016

marcellodesales commented Feb 21, 2016