Calling webkit-patch rebaseline-expectations to rebaseline expectations marked with [ Rebaseline] ends up deleting the entire file. It prints a long list of tests to rebaseline (which, correctly, is the list of test marked as Rebaseline) and then pretends to rebaseline those. In the end the only change it makes though is to _delete_ the TestExpectaitons file.
That sounds bad. :)
wow. does it leave a zero-length/empty file, or actually delete it?
I should also note that when I've seen the "pretends to rebaseline" phenomenon before, it seems to be that the the baselines in your checkout actually match what the bots are producing (and so there's nothing to actually update). In this situation we silently delete the rebaseline line and move on, rather than logging something like "bots match your checkout, nothing to rebaseline". Perhaps the latter would be more useful? Is it possible you were seeing something like this?
(In reply to comment #2) > wow. does it leave a zero-length/empty file, or actually delete it? Zero-length file. > I should also note that when I've seen the "pretends to rebaseline" phenomenon before, it seems to be that the the baselines in your checkout actually match what the bots are producing (and so there's nothing to actually update). In this situation we silently delete the rebaseline line and move on, rather than logging something like "bots match your checkout, nothing to rebaseline". Perhaps the latter would be more useful? Not sure. Executing "webkit-patch rebaseline <single test>" did rebaseline that test (but got the results mixed up between bots) so there certainly was work for it todo.
(In reply to comment #4) > (In reply to comment #2) > > wow. does it leave a zero-length/empty file, or actually delete it? > > Zero-length file. > Okay, well, if the file consisted entire of lines marked with [ Rebaseline ], and the tool thought it rebaselined everything, then this is what I would've expected to happen. Which at least reduces to the problem of "why did it only pretend to rebaseline"? > > Not sure. Executing "webkit-patch rebaseline <single test>" did rebaseline that test (but got the results mixed up between bots) so there certainly was work for it todo. Interesting. Offhand I can't guess why 'rebaseline' would do something but 'rebaseline-expectations' wouldn't, except that the bots were probably in different states. Not sure if that would've mattered. I'll see if I can reproduce something like this. You don't by chance have a copy of the file marked up for rebaselining?
(In reply to comment #5) > (In reply to comment #4) > > (In reply to comment #2) > > > wow. does it leave a zero-length/empty file, or actually delete it? > > > > Zero-length file. > > > > Okay, well, if the file consisted entire of lines marked with [ Rebaseline ], and the tool thought it rebaselined everything, then this is what I would've expected to happen. I would have expected it to remove lines marked as [ Rebaseline ] and leave all other lines intact. While it _did_ remove the ones marked [ Rebaseline ] it also removed all other lines. > > Which at least reduces to the problem of "why did it only pretend to rebaseline"? > > > > > Not sure. Executing "webkit-patch rebaseline <single test>" did rebaseline that test (but got the results mixed up between bots) so there certainly was work for it todo. > > Interesting. Offhand I can't guess why 'rebaseline' would do something but 'rebaseline-expectations' wouldn't, except that the bots were probably in different states. Not sure if that would've mattered. > > I'll see if I can reproduce something like this. You don't by chance have a copy of the file marked up for rebaselining? All TestExpectations lines marked 107771 when synced to r140728.
Oh, sorry, I misread the first comment. I thought you were saying the file *only* contained lines marked with [ Rebaseline ]. It certainly shouldn't delete the other lines. My first thought as to how that could've happened is some failure in serializing the in-memory expectations out that was caught and suppressed incorrectly (causing us to discard all the other expectations). I'll take a look at that.