Planned EWS improvements

# Aakash Jain (a month ago)

I have been making number of improvements to EWS. I also have various planned improvements to EWS. I wanted to reach out to you guys to see if anyone wants me to prioritize any particular improvement(s). If there is any improvement which you want to see and is not listed below, please feel free to let me know. Also most of the queues have been transitioned from old to new EWS and I am working on the remaining ones (jsc, windows and commit-queue).

Here is the list of improvements (in no particular order):

1) Develop a webpage showing summary of EWS builds for a patch. This page would provide the summary of important build-steps, high-level details about the failure (e.g.: name of the tests which failed, or possibly relevant build failure logs), and include link(s) to the Buildbot page(s). This page will open on clicking the status-bubbles (and would be replacement of old EWS status page like webkit-queues.webkit.org/patch/379563/win-ews, webkit-queues.webkit.org/patch/379563/win-ews). Currently clicking the status-bubble opens the Buildbot build page, which contains a lot of infrastructure details, and probably is information-overload for many engineers, so this summary page should help with that. webkit.org/b/197522, webkit.org/b/197522

2) Redesign status-bubble tooltip to include more detailed information about failures (e.g.: each test failure name along-with url to flakiness dashboard, and url to complete results.html file, as suggested by David Kilzer in lists.webkit.org/pipermail/webkit-dev/2019-September/030799.html, lists.webkit.org/pipermail/webkit-dev/2019-September/030799.html). We should also add the tooltip support for iPad/iPhone webkit.org/b/201940, webkit.org/b/201940

3) Add a way to retry a patch in EWS. This would allow engineers to confirm that the failures indicated by EWS aren't flaky/incorrect. Maybe a good place to add the 'retry' button would be the status-bubble's tool-tip (visible only if the bubble is red) webkit.org/b/196599, webkit.org/b/196599

4) Parse the relevant build failure message from build logs (and display in summary page) webkit.org/b/201941, webkit.org/b/201941

5) Style failure should be displayed in-line on the review page along-with the code, just like the reviewer's comments webkit.org/b/202252, webkit.org/b/202252

6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests) webkit.org/b/189206, webkit.org/b/189206, webkit.org/b/201928, webkit.org/b/201928

7) Add commit-queue support for security bugs webkit.org/b/201939, webkit.org/b/201939

8) API tests should upload crashlogs webkit.org/b/201929, webkit.org/b/201929

Thanks Aakash

Contact us to advertise here
# Saam Barati (a month ago)

Thanks for doing this work.

# Ryosuke Niwa (a month ago)

On Fri, Sep 27, 2019 at 7:57 AM Saam Barati <sbarati at apple.com> wrote:

Hi Aakash,

Thanks for doing this work.

On Sep 26, 2019, at 11:27 AM, Aakash Jain <aakash_jain at apple.com> wrote:

Hi Everyone,

I have been making number of improvements to EWS. I also have various planned improvements to EWS. I wanted to reach out to you guys to see if anyone wants me to prioritize any particular improvement(s). If there is any improvement which you want to see and is not listed below, please feel free to let me know. Also most of the queues have been transitioned from old to new EWS and I am working on the remaining ones (jsc, windows and commit-queue).

Here is the list of improvements (in no particular order):

1) Develop a webpage showing summary of EWS builds for a patch. This page would provide the summary of important build-steps, high-level details about the failure (e.g.: name of the tests which failed, or possibly relevant build failure logs), and include link(s) to the Buildbot page(s). This page will open on clicking the status-bubbles (and would be replacement of old EWS status page like webkit-queues.webkit.org/patch/379563/win-ews). Currently clicking the status-bubble opens the Buildbot build page, which contains a lot of infrastructure details, and probably is information-overload for many engineers, so this summary page should help with that. webkit.org/b/197522

2) Redesign status-bubble tooltip to include more detailed information about failures (e.g.: each test failure name along-with url to flakiness dashboard, and url to complete results.html file, as suggested by David Kilzer in lists.webkit.org/pipermail/webkit-dev/2019-September/030799.html). We should also add the tooltip support for iPad/iPhone webkit.org/b/201940

3) Add a way to retry a patch in EWS. This would allow engineers to confirm that the failures indicated by EWS aren't flaky/incorrect. Maybe a good place to add the 'retry' button would be the status-bubble's tool-tip (visible only if the bubble is red) webkit.org/b/196599

This would be amazing. (My 2c: I'd vote for the button being visible without tooltip.)

Yeah, this would be the most useful addition to EWS.

4) Parse the relevant build failure message from build logs (and display in

summary page) webkit.org/b/201941

5) Style failure should be displayed in-line on the review page along-with the code, just like the reviewer's comments webkit.org/b/202252

6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests) webkit.org/b/189206, webkit.org/b/201928

7) Add commit-queue support for security bugs webkit.org/b/201939

8) API tests should upload crashlogs webkit.org/b/201929

Along with this one.

  • R. Niwa
# Simon Fraser (a month ago)

On Sep 27, 2019, at 3:27 AM, Aakash Jain <aakash_jain at apple.com> wrote:

Hi Everyone,

I have been making number of improvements to EWS. I also have various planned improvements to EWS. I wanted to reach out to you guys to see if anyone

Not everyone on this list is a guy.

wants me to prioritize any particular improvement(s). If there is any improvement which you want to see and is not listed below, please feel free to let me know. Also most of the queues have been transitioned from old to new EWS and I am working on the remaining ones (jsc, windows and commit-queue).

Here is the list of improvements (in no particular order):

1) Develop a webpage showing summary of EWS builds for a patch. This page would provide the summary of important build-steps, high-level details about the failure (e.g.: name of the tests which failed, or possibly relevant build failure logs), and include link(s) to the Buildbot page(s). This page will open on clicking the status-bubbles (and would be replacement of old EWS status page like webkit-queues.webkit.org/patch/379563/win-ews, webkit-queues.webkit.org/patch/379563/win-ews). Currently clicking the status-bubble opens the Buildbot build page, which contains a lot of infrastructure details, and probably is information-overload for many engineers, so this summary page should help with that. webkit.org/b/197522, webkit.org/b/197522

2) Redesign status-bubble tooltip to include more detailed information about failures (e.g.: each test failure name along-with url to flakiness dashboard, and url to complete results.html file, as suggested by David Kilzer in lists.webkit.org/pipermail/webkit-dev/2019-September/030799.html, lists.webkit.org/pipermail/webkit-dev/2019-September/030799.html). We should also add the tooltip support for iPad/iPhone webkit.org/b/201940, webkit.org/b/201940

#1

3) Add a way to retry a patch in EWS. This would allow engineers to confirm that the failures indicated by EWS aren't flaky/incorrect. Maybe a good place to add the 'retry' button would be the status-bubble's tool-tip (visible only if the bubble is red) webkit.org/b/196599, webkit.org/b/196599

#2

4) Parse the relevant build failure message from build logs (and display in summary page) webkit.org/b/201941, webkit.org/b/201941

5) Style failure should be displayed in-line on the review page along-with the code, just like the reviewer's comments webkit.org/b/202252, webkit.org/b/202252

6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests) webkit.org/b/189206, webkit.org/b/189206, webkit.org/b/201928, webkit.org/b/201928

7) Add commit-queue support for security bugs webkit.org/b/201939, webkit.org/b/201939

8) API tests should upload crashlogs webkit.org/b/201929, webkit.org/b/201929

Simon

# Konstantin Tokarev (a month ago)

27.09.2019, 08:15, "Simon Fraser" <simon.fraser at apple.com>:

On Sep 27, 2019, at 3:27 AM, Aakash Jain <aakash_jain at apple.com> wrote:

Hi Everyone,

I have been making number of improvements to EWS. I also have various planned improvements to EWS. I wanted to reach out to you guys to see if anyone

Not everyone on this list is a guy.

FWIW, plural form of "guy" is often used as gender-neutral vocative, see e.g. [1,2]

[1] www.merriam-webster.com/dictionary/guy [2] dictionary.cambridge.org/us/dictionary/english/guy

# Aakash Jain (a month ago)

Thank You everyone (who responded) for the feedback.

I will prioritize the features some of you mentioned, especially adding a way to retry the patch.

Thanks Aakash

# Simon Fraser (25 days ago)

On Sep 27, 2019, at 10:27 PM, Konstantin Tokarev <annulen at yandex.ru> wrote:

27.09.2019, 08:15, "Simon Fraser" <simon.fraser at apple.com>:

On Sep 27, 2019, at 3:27 AM, Aakash Jain <aakash_jain at apple.com> wrote:

Hi Everyone,

I have been making number of improvements to EWS. I also have various planned improvements to EWS. I wanted to reach out to you guys to see if anyone

Not everyone on this list is a guy.

FWIW, plural form of "guy" is often used as gender-neutral vocative, see e.g. [1,2]

[1] www.merriam-webster.com/dictionary/guy [2] dictionary.cambridge.org/us/dictionary/english/guy

I apologize to Aakash for calling this out publicly.

However, ‘guys’ clearly has gender-specific roots, and to my ears, it sounds male-biased, There are many other words that can be used in its place which are more inclusive (“folks”, “everyone”, “you all” etc).

Simon

# Adrian Perez de Castro (23 days ago)

Hello Aakash (and everybody else),

On Thu, 26 Sep 2019 14:27:25 -0400, Aakash Jain <aakash_jain at apple.com> wrote:

I have been making number of improvements to EWS. I also have various planned improvements to EWS […]

Thanks once again for your continued work on improving the EWS, it is much appreciated =)

Here is the list of improvements (in no particular order):

[…]

3) Add a way to retry a patch in EWS. This would allow engineers to confirm that the failures indicated by EWS aren't flaky/incorrect. Maybe a good place to add the 'retry' button would be the status-bubble's tool-tip (visible only if the bubble is red) webkit.org/b/196599, webkit.org/b/196599

Being able to retry a patch without having to re-upload it, and more importantly without having to ping another person, would be neat. I don't have a strong opinion about which tasks to prioritize first, tho.

[…]

6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests) webkit.org/b/189206, webkit.org/b/189206, webkit.org/b/201928, webkit.org/b/201928

In this line of adding more things that the EWS could test, among the developers of the GTK/WPE ports we think it would be manageable for us to have bots run API tests. There is some work we need to do, and the intention is to start with GTK first, see how it goes, and then add WPE afterwards. Carlos opened a bug for this, I took the liberty of CC'ing you assuming that you might want to at least keep tabs on it (feel free to unsubscribe):

bugs.webkit.org/show_bug.cgi?id=202361

Best

# Aakash Jain (23 days ago)

On Sep 30, 2019, at 9:32 AM, Adrian Perez de Castro <aperez at igalia.com> wrote:

Hello Aakash (and everybody else),

On Thu, 26 Sep 2019 14:27:25 -0400, Aakash Jain <aakash_jain at apple.com> wrote:

I have been making number of improvements to EWS. I also have various planned improvements to EWS […]

Thanks once again for your continued work on improving the EWS, it is much appreciated =)

Thank You for the feedback. I am hoping that Improved EWS would increase overall team's productivity as people have to deal less with build/test breakages and rollouts.

Here is the list of improvements (in no particular order):

[…]

3) Add a way to retry a patch in EWS. This would allow engineers to confirm that the failures indicated by EWS aren't flaky/incorrect. Maybe a good place to add the 'retry' button would be the status-bubble's tool-tip (visible only if the bubble is red) webkit.org/b/196599, webkit.org/b/196599

Being able to retry a patch without having to re-upload it, and more importantly without having to ping another person, would be neat. I don't have a strong opinion about which tasks to prioritize first, tho.

[…]

6) Add more test-suites to EWS (e.g.: LLDB tests, resultsdbpy tests) webkit.org/b/189206, webkit.org/b/189206, webkit.org/b/201928, webkit.org/b/201928

In this line of adding more things that the EWS could test, among the developers of the GTK/WPE ports we think it would be manageable for us to have bots run API tests. There is some work we need to do, and the intention is to start with GTK first, see how it goes, and then add WPE afterwards. Carlos opened a bug for this, I took the liberty of CC'ing you assuming that you might want to at least keep tabs on it (feel free to unsubscribe):

bugs.webkit.org/show_bug.cgi?id=202361

Sure, I will work with you and Carlos on this.

# Aakash Jain (a day ago)

I am very happy to announce that I have added the ability to retry the patches in EWS :)

From now on, for any patch, whenever any of the EWS status-bubble is red, you will see another button named 'Retry failed builds' next to the status-bubbles. Clicking on 'Retry failed builds' button would cause all the failed EWS builds for that patch to be retried. Please click on this button only when needed, as unnecessary retries might put load on the system and others patches might have to wait longer.

As always, please feel free to let me know if you notice any issue or have any feedback.

Known UI issues: bugs.webkit.org/show_bug.cgi?id=203249, bugs.webkit.org/show_bug.cgi?id=203249

Thanks Aakash

Want more features?

Request early access to our private beta of readable email premium.