In this commit, we effectively revert #699 by allow clients to request
filter for up to 1k consecutive blocks. Testing in the field has shown
that applications are able to reduce perceived latency from syncing to
full functionality after an app has been offline for several days by
batching requests for filters. A value of 100 would mean each additional
day behind adds an additional round trip, resulting in 10s of
seconds of lag after just a few days of being offline. A value of ~1k
allows implementations to catch up with roughly a week's worth of
filters in a single round trip.
In this commit, we clarify how we handle `OP_RETURN` outputs for regular
filters. The prior language was a bit ambiguous, so we hope to make it
as explicit as possible.
In this commit, we add a new test case for a filter built from a block
that has a transaction with an OP_RETURN which isn't followed by only
push data items. The prior implementation for btcd (which was used to
generated these test vectors), had a stricter check which caused it to
add extra items to the filter. We also add a case of a block that has a
single coinbase transaction, with that transaction having only an
OP_RETURN output. As a result, that filter will be "empty", and is
signalled by by a single zero (0x00) byte.
In order to make building the code that makes the test vectors
reproducible, we've added go.mod and go.sum files as well.
In this commit, we simplify the code that generates the test vectors to
only generate filters for a target fp of 19, and also only for the
regular filter, as it's the only filter type currently defined.
The test vectors have also been updated to include the previous output
scripts for all input within a block as these are now required to
construct the regular filter.
Finally, the generation code has been updated to properly fetch the
previous input scripts to the generation code can verify the filter it
generates manually against the end server.
In this commit, we modify regular filter construction slightly. Rather
than including each pushed data in the script, we instead just include
the script directly, which will eventually be hashed. The rationale for
doing this is two-fold:
* Most scripts today and in the foreseeable future will just be a
commitment.
* Including only the script itself and not the hash of the script
reduces the worst case filter size. Otherwise, an attacker could
include a bunch of 2 byte push datas and blow up the filter size for
all nodes.
Olaoluwa Osuntokun