Git Bisect: The go-to tool for answering the "when did this stop working" question

Submitted by Barrett on Sun, 03/13/2016 - 21:24
Git Bisect: The go-to tool for answering the "when did this stop working" question

Continuous integration and automated testing can significantly reduce the odds of regressions, but eventually, every project will fnd themselves facing a feature that used to work and no longer does.  When that time comes for you, I recommend git bisect.

As the name suggests, git bisect cuts a commit range in half over and over until the commit in which the regression occurred is identified.  Just start it up, give it the SHA of a bad commit, the SHA of a known good commit, and for each cadidate commit it identifies tell bisect if it's good or bad.

To illustrate, a client with strict user privacy policies was interested in the reCAPTCHA module but concerned about the data that is sent back to Google.  Digging through the issue queues, they found an old issue marked as "Fixed" that added code to stop the setting of 3rd-party cookies, but that functionality isn't in the module any more.  Nothing in the log messages explicitly mentioned removing the code but I knew the SHA of a "good" commit and what to look for in each commit to see if the code was still there, so it was bisect time.

$ git bisect start 
$ git bisect bad 
$ git bisect good 0177a3 
Bisecting: 59 revisions left to test after this (roughly 6 steps) [24487c097fcbf7b686574c168b1c5a815bf96475] Change wording 

$ grep -in recaptcha_nocookies 
$ git bisect bad 
Bisecting: 29 revisions left to test after this (roughly 5 steps) [410cc62fcbf0870fb83a2573baf90c98987d3bc2] Issue #1959274: Recaptcha misspelled in recaptcha.js. 

$ grep -in recaptcha_nocookies 
41: $form['recaptcha_nocookies'] = array( 
44: '#default_value' => variable_get('recaptcha_nocookies', FALSE), 
$ git bisect good 
Bisecting: 14 revisions left to test after this (roughly 4 steps) [f3876faf1fa26f48049e3b0fa20a74da2d6112ad] Issue #2407929 by drupalexio: Declare api.js from as external. 

As illustrated in the command line output above, you start off bisect using the appropriately named `git bisect start`, telling it the current commit is bad, and giving it the SHA of the commit in which the patch was added as a good commit. Bisect then halved the intervening commits and picked the middle one. I tested the commit by grepping for the presence of the option field in, told bisect that commit was bad, and the processes repeated until finally I tested the last "middle point" commit and bisect came back with my answer.  Apparently, in 2014 the module went through a major restructure and with it went the nocookie option.

$ git bisect good
e57ea9bf1f8be27bab4f76333e3ea37923f68ca8 is the first bad commit
commit e57ea9bf1f8be27bab4f76333e3ea37923f68ca8
Author: diolan 
Date:   Tue Dec 9 17:37:24 2014 -0500

Issue #2386815 by Liam Morland, diolan: Copy google_captcha module and rename to recaptcha.
:100644 100644 56b7d18783d1c5c90540f9f87442fd65ea74a2fb 82f1b08071522032e72d3970e1ddf50225d42a71 M  README.txt
:000000 040000 0000000000000000000000000000000000000000 702db26b7720bd69a4389a2399867487643667bd A  ReCAPTCHA
:040000 000000 ff906537502e2a72c116337843f044866717f231 0000000000000000000000000000000000000000 D  recaptcha-php-1.11
:100644 100644 5f13e38c9d89e33891314142035dea7fdbb97fb6 b6226020ce75ad36a5c707a11aeb0d48491c8f59 M
:100644 100644 1300d7fbb6539b6dad7885dc3c200c28a62b12b9 a0232a5f4c88f4555e041e0ec66f2b72d60dd445 M
:100644 100644 4b7927f644ab99da4976a8ff18e10a9c27598429 ae9b3eed2fd62c9f024831cf34263e34f8718583 M  recaptcha.install
:100644 000000 a0adb80d30402163b98497efae5ab5c279db0ce1 0000000000000000000000000000000000000000 D  recaptcha.js
:100644 100644 642681921045429e4de1da8aef0b648cca96dbe3 6903e91f6a47c3b5895d27329657b77bf98f5dcd M  recaptcha.module
:100644 000000 9fc4d0f25661964521d5b85560079a2c8550e740 0000000000000000000000000000000000000000 D
:100644 000000 9fc5e9c958571ceae1cba1354102c2b244978aca 0000000000000000000000000000000000000000 D  recaptcha_mailhide.module
Barrett Sun, 03/13/2016 - 21:24