Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: rustyrussell/pettycoin
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 5335d49e011f
Choose a base ref
...
head repository: rustyrussell/pettycoin
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: 49416994d886
Choose a head ref
  • 15 commits
  • 45 files changed
  • 1 contributor

Commits on Oct 25, 2014

  1. log: use single logging record for all logs.

    The current logs are separate, ie. JSON-RPC, one per peer peer,
    generator and core.  But the core log is usually related to what's
    happening in the other logs, so it's hard to interpret them separately.
    
    Use a single log backend for all of them.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    47dfcb7 View commit details
  2. log: add a per-line iterator.

    This will be used for JSON log output.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    238463d View commit details
  3. jsonrpc: add getlog command.

    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    0633110 View commit details
  4. jsonrpc: dev-detachedblocks command

    Useful for debugging: what blocks do we have where we don't know
    the parent?
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    e707244 View commit details
  5. json: vastly speed up json creation.

    Now we dump large JSON (ie. logs) our multiple strlen() calls really
    hurt us.  We could get even more efficient by not continually realloc()
    to extend the buffer, but this patch abstracts the type so we could
    do that in future with minimal disruption.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    66ae498 View commit details
  6. pettycoin-query: speed up handling of large results.

    The JSON parser is simply too slow, so add a hacky fastpath if it's
    in the format we expect.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    994a7fa View commit details
  7. peer: continue to send blocks while syncing.

    While tracking down bug #32 (where nodes fail to sync) I found that mostly
    my peer failed to sync.  The reason is that:
    
    1) By the time we sync up to the initial welcome block, there are other
       blocks which have been generated.
    2) Newly generated blocks aren't sent to us while we're syncing.
    
    This fixes the sending path, but I'm still investigating why we didn't
    get the predecessors after syncing.     
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    132b471 View commit details
  8. todo: wake peer when we complete a todo.

    I noticed that my peer got stuck syncing.  It had a huge TODO list, but it
    wasn't sending them out.  This is because we hit MAX_REQUESTS and then don't
    wake up again when we process a request.  Fix this.
    
    We still have a problem where our todo list gets backlogged with requests
    for the children of every single block, and takes quite a while to clear.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    7d26548 View commit details
  9. welcome: use normal recv_block path for welcome block.

    This puts all the logic in one place.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    376f6da View commit details
  10. welcome: use peer_read_packet().

    This is one case where we read into something other than peer->incoming;
    unify that.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    3279efc View commit details
  11. welcome: duplicate the welcome packet.

    If all goes well, the welcome packet gets owned by "state", which
    outlives the peer.  But if it becomes a detached packet, it is owned
    by that detached_block structure, and then is later discovered to be
    bogus, we'll free it before peer.  In that case we'll queue an error
    packet, but a correctly timed input packet could cause us to dereference
    peer->welcome before that closes the connection.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    0c5e506 View commit details
  12. todo: fix uninitialized 'unused' field in PROTOCOL_PKT_GET_SHARD pack…

    …ets.
    
    Valgrind says:
    
    ==2904== Syscall param write(buf) points to uninitialised byte(s)
    ==2904==    at 0x55053B0: __write_nocancel (syscall-template.S:81)
    ==2904==    by 0x432EA5: do_write (io.c:170)
    ==2904==    by 0x433523: do_plan (io.c:368)
    ==2904==    by 0x4335BD: io_ready (io.c:389)
    ==2904==    by 0x434814: io_loop (poll.c:295)
    ==2904==    by 0x408D0B: main (pettycoin.c:375)
    ==2904==  Address 0xaf1c41a is 74 bytes inside a block of size 104 alloc'd
    ==2904==    at 0x4C2AB80: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
    ==2904==    by 0x42AD59: allocate (tal.c:230)
    ==2904==    by 0x42B23A: tal_alloc_ (tal.c:390)
    ==2904==    by 0x42B444: tal_alloc_arr_ (tal.c:447)
    ==2904==    by 0x42C137: tal_dup_ (tal.c:806)
    ==2904==    by 0x41ADF4: get_todo_pkt (todo.c:356)
    ==2904==    by 0x40D04F: plan_output (peer.c:465)
    ==2904==    by 0x432ADD: next_plan (io.c:58)
    ==2904==    by 0x433611: io_do_always (io.c:398)
    ==2904==    by 0x4345B6: handle_always (poll.c:218)
    ==2904==    by 0x434651: io_loop (poll.c:244)
    ==2904==    by 0x408D0B: main (pettycoin.c:375)
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    311e141 View commit details
  13. horizon: routines to check horizon.

    Corner cases like < vs <= are important, so centralize the logic
    inside a single helper.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    c4daa69 View commit details
  14. create_refs, inputs_refs: use horizon helpers.

    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    8c8e200 View commit details
  15. recv_block: when syncing, don't ask about blocks below horizon.

    This currently fails because it requires that we adjust the heuristic
    for "known" blocks to only go back to the horizon, so we still ask for
    the transactions (if any), just not the children.
    
    Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
    rustyrussell committed Oct 25, 2014
    Copy the full SHA
    4941699 View commit details
Showing with 1,054 additions and 487 deletions.
  1. +1 −1 Makefile.in
  2. +3 −3 chain.c
  3. +4 −4 create_refs.c
  4. +28 −0 detached_block.c
  5. +2 −4 generating.c
  6. +2 −2 getinfo.c
  7. +1 −1 getpeerinfo.c
  8. +1 −1 gettransaction.c
  9. +17 −0 horizon.c
  10. +16 −0 horizon.h
  11. +3 −2 input_refs.c
  12. +111 −51 json.c
  13. +27 −19 json.h
  14. +4 −4 json_add_tx.c
  15. +2 −1 json_add_tx.h
  16. +125 −14 jsonrpc.c
  17. +2 −1 jsonrpc.h
  18. +15 −14 listtransactions.c
  19. +162 −87 log.c
  20. +37 −13 log.h
  21. +10 −4 minimal_log.c
  22. +36 −66 peer.c
  23. +1 −2 peer.h
  24. +27 −7 pettycoin-query.c
  25. +15 −14 pettycoin.c
  26. +40 −3 recv_block.c
  27. +7 −0 recv_block.h
  28. +1 −1 sendrawtransaction.c
  29. +4 −7 state.c
  30. +1 −2 state.h
  31. +117 −0 test/run-01-horizon.c
  32. +12 −11 test/run-01-json.c
  33. +28 −2 test/run-01-jsonrpc.c
  34. +14 −13 test/run-02-prev_txhashes.c
  35. +20 −13 test/run-03-check_block.c
  36. +15 −13 test/run-04-create_normal_transaction.c
  37. +15 −13 test/run-05-create_to_gateway_transaction.c
  38. +20 −13 test/run-05-proof.c
  39. +16 −12 test/run-06-log.c
  40. +20 −13 test/run-07-block_swizzle.c
  41. +14 −13 test/run-08-simple-chain.c
  42. +14 −13 test/run-09-chain.c
  43. +14 −12 test/run-10-doublespend.c
  44. +7 −6 test/run-20-gateway.c
  45. +23 −12 todo.c
2 changes: 1 addition & 1 deletion Makefile.in
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@
# blackbox-check: run the blackbox tests
# update-mocks: regenerate the mocks for the unit tests.

PETTYCOIN_OBJS := block.o check_block.o check_tx.o difficulty.o shadouble.o timestamp.o gateways.o hash_tx.o pettycoin.o merkle_txs.o merkle_recurse.o tx_cmp.o genesis.o marshal.o hash_block.o prev_txhashes.o state.o tal_packet.o dns.o netaddr.o peer.o peer_cache.o pseudorand.o welcome.o log.o generating.o blockfile.o pending.o log_helper.o txhash.o signature.o proof.o chain.o features.o todo.o base58.o sync.o create_refs.o shard.o packet_io.o tx.o complain.o block_shard.o recv_block.o input_refs.o peer_wants.o inputhash.o tx_in_hashes.o merkle_hashes.o recv_tx.o reward.o recv_complain.o json.o jsonrpc.o getinfo.o ecode_names.o sendrawtransaction.c pettycoin_dir.o pkt_names.o hex.o listtransactions.o json_add_tx.o gettransaction.o prev_blocks.o detached_block.o getpeerinfo.o
PETTYCOIN_OBJS := block.o check_block.o check_tx.o difficulty.o shadouble.o timestamp.o gateways.o hash_tx.o pettycoin.o merkle_txs.o merkle_recurse.o tx_cmp.o genesis.o marshal.o hash_block.o prev_txhashes.o state.o tal_packet.o dns.o netaddr.o peer.o peer_cache.o pseudorand.o welcome.o log.o generating.o blockfile.o pending.o log_helper.o txhash.o signature.o proof.o chain.o features.o todo.o base58.o sync.o create_refs.o shard.o packet_io.o tx.o complain.o block_shard.o recv_block.o input_refs.o peer_wants.o inputhash.o tx_in_hashes.o merkle_hashes.o recv_tx.o reward.o recv_complain.o json.o jsonrpc.o getinfo.o ecode_names.o sendrawtransaction.c pettycoin_dir.o pkt_names.o hex.o listtransactions.o json_add_tx.o gettransaction.o prev_blocks.o detached_block.o getpeerinfo.o horizon.o
PETTYCOIN_GENERATE_OBJS := pettycoin-generate.o merkle_hashes.o merkle_recurse.o hash_tx.o tx_cmp.o shadouble.o marshal.o minimal_log.o tal_packet.o hex.o
MKGENESIS_OBJS := mkgenesis.o shadouble.o hash_block.o merkle_hashes.o merkle_recurse.o minimal_log.o
SIZES_OBJS := sizes.o
6 changes: 3 additions & 3 deletions chain.c
Original file line number Diff line number Diff line change
@@ -465,7 +465,7 @@ void update_block_ptrs_invalidated(struct state *state,
wake_peers(state);
}

static void json_add_tx(char **response,
static void json_add_tx(struct json_result *response,
const struct block_shard *s,
unsigned int txoff)
{
@@ -505,7 +505,7 @@ static void json_add_tx(char **response,

static char *json_getblock(struct json_connection *jcon,
const jsmntok_t *params,
char **response)
struct json_result *response)
{
struct protocol_block_id sha;
const struct block *b, *b2;
@@ -581,7 +581,7 @@ const struct json_command getblock_command = {

static char *json_getblockhash(struct json_connection *jcon,
const jsmntok_t *params,
char **response)
struct json_result *response)
{
jsmntok_t *height;
unsigned int h;
8 changes: 4 additions & 4 deletions create_refs.c
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
#include "block.h"
#include "chain.h"
#include "create_refs.h"
#include "horizon.h"
#include "state.h"
#include "timestamp.h"
#include "tx.h"
@@ -27,10 +28,9 @@ static bool resolve_input(struct state *state,
if (!te)
return false;

/* Don't include any transactions within 1 hour of cutoff. */
if (block_timestamp(&te->u.block->bi)
+ PROTOCOL_TX_HORIZON_SECS(state->test_net) - CLOSE_TO_HORIZON
< current_time())
/* Don't include any transactions expiring the next 1 hour. */
if (block_expired_by(block_expiry(state, &te->u.block->bi),
current_time() + CLOSE_TO_HORIZON))
return false;

/* Add offset: it might be going to go into *next* block */
28 changes: 28 additions & 0 deletions detached_block.c
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#include "block.h"
#include "detached_block.h"
#include "jsonrpc.h"
#include "protocol_net.h"
#include "recv_block.h"
#include "state.h"
@@ -64,3 +65,30 @@ void add_detached_block(struct state *state,
bd->pkt_ctx = tal_steal(bd, pkt_ctx);
list_add(&state->detached_blocks, &bd->list);
}

static char *json_detachedblocks(struct json_connection *jcon,
const jsmntok_t *params,
struct json_result *response)
{
struct detached_block *bd;

json_array_start(response, NULL);
list_for_each(&jcon->state->detached_blocks, bd, list) {
json_object_start(response, NULL);
json_add_num(response, "height",
le32_to_cpu(bd->bi.hdr->height));
json_add_double_sha(response, "sha", &bd->sha.sha);
json_add_double_sha(response, "prev",
&block_prev(&bd->bi, 0)->sha);
json_object_end(response);
}
json_array_end(response);
return NULL;
}

const struct json_command detachedblocks_command = {
"dev-detachedblocks",
json_detachedblocks,
"Get all detached blocks",
"Returns height, sha and prev for each detached block"
};
6 changes: 2 additions & 4 deletions generating.c
Original file line number Diff line number Diff line change
@@ -247,7 +247,6 @@ static void exec_generator(struct generator *gen)
char nonce[14 + 1];
int i;
const struct block *last;
char log_prefix[40];

/* FIXME: This is where we increment shard_order if voted! */
gen->shard_order = gen->state->longest_knowns[0]->bi.hdr->shard_order;
@@ -295,9 +294,8 @@ static void exec_generator(struct generator *gen)
exit(127);
}

sprintf(log_prefix, "Generator %u:", gen->pid);
gen->log = new_log(gen, gen->state->log,
log_prefix, gen->state->log_level, GEN_LOG_MAX);
gen->log = new_log(gen, gen->state->lr, "%sGenerator %u:",
log_prefix(gen->state->log), gen->pid);
log_debug(gen->log, "Running '%s' '%s' '%s' '%s' %s' '%s' '%s' '%s'",
gen->state->generator,
fees_to,
4 changes: 2 additions & 2 deletions getinfo.c
Original file line number Diff line number Diff line change
@@ -4,7 +4,7 @@
#include "state.h"
#include "todo.h"

static void json_add_block(char **response, const char *fieldname,
static void json_add_block(struct json_result *response, const char *fieldname,
const struct block *block)
{
json_object_start(response, fieldname);
@@ -16,7 +16,7 @@ static void json_add_block(char **response, const char *fieldname,

static char *json_getinfo(struct json_connection *jcon,
const jsmntok_t *params,
char **response)
struct json_result *response)
{
size_t i, num_todo, num_peer_todo, num_peers;
struct todo_request *todo;
2 changes: 1 addition & 1 deletion getpeerinfo.c
Original file line number Diff line number Diff line change
@@ -27,7 +27,7 @@

static char *json_getpeerinfo(struct json_connection *jcon,
const jsmntok_t *params,
char **response)
struct json_result *response)
{
struct peer *peer;
struct state *state = jcon->state;
2 changes: 1 addition & 1 deletion gettransaction.c
Original file line number Diff line number Diff line change
@@ -9,7 +9,7 @@

static char *json_gettransaction(struct json_connection *jcon,
const jsmntok_t *params,
char **response)
struct json_result *response)
{
struct protocol_tx_id txhash;
const jsmntok_t *txid;
17 changes: 17 additions & 0 deletions horizon.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#include "block_info.h"
#include "horizon.h"
#include "state.h"
#include "timestamp.h"

/* When do transactions in this block expire? */
u32 block_expiry(struct state *state, const struct block_info *bi)
{
return block_timestamp(bi)
+ PROTOCOL_TX_HORIZON_SECS(state->test_net);
}

bool block_expired_by(u32 expires, u32 now)
{
/* We allow line-balls: ie. if exactly on horizon, it's *not* expired. */
return expires < now;
}
16 changes: 16 additions & 0 deletions horizon.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#ifndef PETTYCOIN_HORIZON_H
#define PETTYCOIN_HORIZON_H
#include "config.h"
#include <ccan/short_types/short_types.h>
#include <stdbool.h>

struct state;
struct block_info;

/* When do transactions in this block expire? */
u32 block_expiry(struct state *state, const struct block_info *bi);

/* Has this expiry time mean it's expired? */
bool block_expired_by(u32 expires, u32 now);

#endif /* PETTYCOIN_HORIZON_H */
5 changes: 3 additions & 2 deletions input_refs.c
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
#include "block_shard.h"
#include "chain.h"
#include "horizon.h"
#include "input_refs.h"
#include "shard.h"
#include "tx.h"
@@ -15,8 +16,8 @@ static enum protocol_ecode check_ref(struct state *state,
return PROTOCOL_ECODE_REF_BAD_BLOCKS_AGO;

/* Beyond horizon? */
if (block_timestamp(&b->bi) + PROTOCOL_TX_HORIZON_SECS(state->test_net)
< block_timestamp(&block->bi))
if (block_expired_by(block_expiry(state, &b->bi),
block_timestamp(&block->bi)))
return PROTOCOL_ECODE_REF_BAD_BLOCKS_AGO;

if (le16_to_cpu(ref->shard) >= block_num_shards(&b->bi))
Loading