When fetching content from a remote, 'git index-pack' processes the packfile content, storing a packfile appropriate for on-disk storage and a pack-index helping to perform random-access into that packfile. To help with compression, the packfile sent over the wire can use REF_DELTAs in addition to OFS_DELTAs to refer to objects that are already known to exist in the client's repository. REF_DELTAs can also refer to objects within the packfile, though this is not typically done. Because this inter-pack REF_DELTA is not a typical data shape, a latent bug has been waiting that causes 'git index-pack' to die() even on legitimate packfile content that it could resolve. This series resolves this problem while also creating a test helper for constructing packfiles with specific objects represented in specific types of deltas and in a given order. This should make it easier to create test cases like this in the future instead of updating t/lib-pack.sh through other means. Updates in V2 ============= * Fixed a memory leak in the test helper. * The test helper has a better CLI that makes use of the parse-options library. * The test script skips the in file and instead feeds the input directly to the test helper. Thanks, -Stolee Derrick Stolee (3): test-tool: add pack-deltas helper t5309: create failing test for 'git index-pack' index-pack: allow revisiting REF_DELTA chains Makefile | 1 + builtin/index-pack.c | 58 ++++++++------ t/helper/meson.build | 1 + t/helper/test-pack-deltas.c | 148 +++++++++++++++++++++++++++++++++++ t/helper/test-tool.c | 1 + t/helper/test-tool.h | 1 + t/t5309-pack-delta-cycles.sh | 34 +++++++- 7 files changed, 216 insertions(+), 28 deletions(-) create mode 100644 t/helper/test-pack-deltas.c base-commit: 4bbb303af69990ccd05fe3a2eb58a1ce036f8220 Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1906%2Fderrickstolee%2Findex-pack-ref-deltas-v2 Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1906/derrickstolee/index-pack-ref-deltas-v2 Pull-Request: https://github.com/gitgitgadget/git/pull/1906 Range-diff vs v1: 1: 5d4beb202d6 ! 1: 41aac8e782f test-tool: add pack-deltas helper @@ t/helper/test-pack-deltas.c (new) +#include "hex.h" +#include "pack.h" +#include "pack-objects.h" ++#include "parse-options.h" +#include "setup.h" +#include "strbuf.h" +#include "string-list.h" + -+static const char usage_str[] = "test-tool pack-deltas <n>"; ++static const char *usage_str[] = { ++ "test-tool pack-deltas --num-objects <num-objects>", ++ NULL ++}; + +static unsigned long do_compress(void **pptr, unsigned long size) +{ @@ t/helper/test-pack-deltas.c (new) + +int cmd__pack_deltas(int argc, const char **argv) +{ -+ int N; ++ int num_objects = -1; + struct hashfile *f; + struct strbuf line = STRBUF_INIT; ++ struct option options[] = { ++ OPT_INTEGER('n', "num-objects", &num_objects, N_("the number of objects to write")), ++ OPT_END() ++ }; + -+ if (argc != 2) { -+ usage(usage_str); -+ return -1; -+ } ++ argc = parse_options(argc, argv, NULL, ++ options, usage_str, 0); + -+ N = atoi(argv[1]); ++ if (argc || num_objects < 0) ++ usage_with_options(usage_str, options); + + setup_git_directory(); + + f = hashfd(the_repository->hash_algo, 1, "<stdout>"); -+ write_pack_header(f, N); ++ write_pack_header(f, num_objects); + + /* Read each line from stdin into 'line' */ + while (strbuf_getline_lf(&line, stdin) != EOF) { @@ t/helper/test-pack-deltas.c (new) + if (get_oid_hex(base_oid_str, &base_oid)) + die("invalid object: %s", base_oid_str); + } ++ string_list_clear(&items, 0); + + if (!strcmp(type_str, "REF_DELTA")) + write_ref_delta(f, &content_oid, &base_oid); 2: a9430447641 ! 2: 53a990e69ea t5309: create failing test for 'git index-pack' @@ t/t5309-pack-delta-cycles.sh: test_expect_success 'failover to a duplicate objec + C=$(git -C server rev-parse HEAD~2^{tree}) && + git -C server reset --hard HEAD~1 && + -+ cat >in <<-EOF && ++ test-tool -C server pack-deltas --num-objects=2 >thin.pack <<-EOF && + REF_DELTA $A $B + REF_DELTA $B $C + EOF + -+ test-tool -C server pack-deltas 2 <in >thin.pack && -+ + git clone "file://$(pwd)/server" client && + ( + cd client && 3: 27d36402fe9 = 3: 1358039b2f3 index-pack: allow revisiting REF_DELTA chains -- gitgitgadget
