Recent EWS improvements

# Aakash Jain (3 months ago)

I just wanted to update everyone with the recent improvements I have made to new EWS. As always, please feel encouraged to provide any feedback (either by filing bugs or contacting me directly).

New Features: EWS status-bubble now display position in queue while patch is waiting to be processed Added webkitpy and bindings-tests EWS (moved from old to new EWS) Status bubbles for webkitpy and bindings-tests EWS now display the exact test failures in hover-over message (webkit.org/b/197395, webkit.org/b/197395, webkit.org/b/197423, webkit.org/b/197423)

Added support for 'new EWS' in webkit-patch tool Added 'EWS Build Archives' (similar to 'WebKit Build Archives' webkit.org/blog/7978/introducing-webkit-build-archives, webkit.org/blog/7978/introducing-webkit-build-archives). For every patch uploaded to Bugzilla, EWS builders build the patch for various platforms (currently macOS and iOS) and upload the archives to S3. These archives are available to download by anyone (for 14 days). The S3 URL is in corresponding build (e.g.: notice 'uploaded archive' link in ews-build.webkit.org/#/builders/7/builds/2477, ews-build.webkit.org/#/builders/7/builds/2477). So, if for any reason, you want to get a built archive for your patch, you can simply upload the patch to Bugzilla. (Note that if there is interest in this, we can enhance it further)

Infrastructure Improvements: Flakiness in API tests has been reduced (thanks to many WebKit developers) Infrastructure improvements to prevent build failure due to "worker not pinged" (e.g.: ews-build.webkit.org/#/builders/9/builds/332, ews-build.webkit.org/#/builders/9/builds/332)

New EWS polls bugzilla more frequently webkit.org/b/197138, webkit.org/b/197138

Configured DEBUG mode appropriately for Production and Development env webkit.org/b/197700, webkit.org/b/197700

Ensured that Buildbot worker logs are not lost on restarting worker Do not run clean build by default on EWS builders (to improve efficiency) webkit.org/b/196897, webkit.org/b/196897

build.webkit.org build.webkit.org and ews-build.webkit.org ews-build.webkit.org starting sharing code (although very little as of now, however the plan is to share more code)

Added migrations file to repository webkit.org/b/197729, webkit.org/b/197729

Added EWS bots information to Internal scripts to easily monitor bots Added more unit-tests

Bug fixes: Clicking 'submit to new ews' doesn't reload status-bubble webkit.org/b/196675, webkit.org/b/196675

Clicking on white bubble navigates to page with only bubbles webkit.org/b/197520, webkit.org/b/197520

Submit to EWS buttons are not aligned properly with status-bubbles webkit.org/b/197139, webkit.org/b/197139

Status bubble should turn orange when any build step fails webkit.org/b/197812, webkit.org/b/197812

Handle bug titles with unicode characters webkit.org/b/196802, webkit.org/b/196802

Scripts using Buildbot API have CORS error webkit.org/b/196709, webkit.org/b/196709

PrintConfiguration should display Xcode version instead of SDKVersion webkit.org/b/196780, webkit.org/b/196780

Trigger queues only after uploading the archive webkit.org/b/197180, webkit.org/b/197180

Do not upload archive when Compile Fails webkit.org/b/196674, webkit.org/b/196674

Exception while loading status-bubble when no build step has started webkit.org/b/196676, webkit.org/b/196676

Use singular verb in failure description in case of single api test failure webkit.org/b/197013, webkit.org/b/197013

EWS should clearly indicate flaky test failures webkit.org/b/196947, webkit.org/b/196947

Use explicit imports instead of wildcard imports webkit.org/b/197194, webkit.org/b/197194

New EWS: patches on recently added queues listed as #1 for older bugs webkit.org/b/197496, webkit.org/b/197496

Improved summary text for various build steps

Interesting info: Since last month, 'EWS for API tests' prevented API test breakage on 50+ patches (ews-build.webkit.org/api/v2/builders/3/builds?state_string__contains=new%20API%20Test&property=bug_id&order=-number, ews-build.webkit.org/api/v2/builders/3/builds?state_string__contains=new%20API%20Test&property=bug_id&order=-number).

Thanks Aakash

Contact us to advertise here
# Aakash Jain (2 months ago)

I want to update everyone with the further improvements I have made to new EWS. As always, please feel encouraged to provide any feedback (either by filing bugs or contacting me directly).

New Features:

  • Launched 'Security EWS'
  • Added iOS-12 Builder queue on new EWS (moved from old to new EWS)
  • Added WPE and GTK queues on new EWS (moved from old to new EWS)
  • New EWS can now process large patches (larger than 640kb) webkit.org/b/198851, webkit.org/b/198851

Infrastructure Improvements:

Bug fixes:

Thanks Aakash

# Aakash Jain (6 days ago)

I want to update everyone with the further improvements I have made to new EWS. As always, please feel free to provide any feedback (either by filing bugs or contacting me directly).

New Features:

  • Moved following queues to from old to new EWS: • iOS 12 Tests EWS • macOS WK1 and WK2 Release Tests EWS • macOS WK1 Debug Tests EWS • GTK and WPE EWS • WinCairo EWS • Style (and watchlist) EWS

  • Added a new 'services' EWS to run Buildbot unit tests webkit.org/b/199994

  • Separate status bubbles for builders and layout-testers

Infrastructure Improvements:

  • EWS is now able to automatically recover (and retry the build) in case of certain git and network errors webkit.org/b/199722
  • Some EWS bots now reboot periodically to ensure that start from clean state
  • Added KillOldProcesses step before running API or Layout tests to increase robustness webkit.org/b/199592
  • 'view layout test results' option is now displayed next to layout-test build step webkit.org/b/200048
  • EWS now displays pre-existing Layout test failure names in the build summary webkit.org/b/199941
  • Machine uptime is now reported in PrintConfiguration step webkit.org/b/200812
  • Added EWS Buildbot support to Internal scripts to easily find/reconfigure bots
  • Flakiness in API tests has been further reduced (thanks to multiple fixes by WebKit developers)
  • Added more unit tests

Bug fixes:

  • EWS status-bubbles are sometimes multi-row with scroll-bar webkit.org/b/199939
  • EWS: Cannot see build status page when patch is waiting for tester webkit.org/b/200333
  • EWS: Do not append additional '(failure)' string at the end of custom failure message in EWS Buildbot webkit.org/b/201140
  • [ews] Status bubble should display only important messages in pop-over webkit.org/b/201308

Only remaining queues on old EWS are windows, jsc and commit-queue, which I will be working on next.

Thanks Aakash

# Adrian Perez de Castro (11 hours ago)

Hello Aakash, WebKittens,

On Thu, 29 Aug 2019 18:01:56 -0400, Aakash Jain <aakash_jain at apple.com> wrote:

I want to update everyone with the further improvements I have made to new EWS. As always, please feel free to provide any feedback (either by filing bugs or contacting me directly).

New Features:

[...]

Thanks a ton for all the improvements in the EWS! Since we moved the GTK/EWS builders to the new queues they have been faster and more reliable. After solving a couple of minor hiccups right after the switch, the builders have not needed any manual intervention since (with the old system we would need to clean stray files from old builds now and then, but not anymore!)

\o/

Only remaining queues on old EWS are windows, jsc and commit-queue, which I will be working on next.

Today Xan López and I talked a bit about the JSCOnly queue. We have a many builders targeting different architectures, which we would like to have moved to the new system.

In the case of the JSC port, instead of grouping all of the builders under a single “jsc” status bubble it would be better to have a bubble for each target architecture e.g. “jsc-armv7”, “jsc-arm64”, and so on.

The motivation is that we think it makes sense to that the system considers that a patch cannot be landed if it breaks any of the supported JSC architectures. With a single status bubble grouping them, we can have situations where a builder for one architecture passes the build+checks but the patch may still break some other architecture (e.g. a patch that touches ARM code generation gets picked by a MIPS builder).

Is the above something that can supported? Let us know if you need more information and/or some support from our side for this.

Best

# Aakash Jain (6 hours ago)

On Sep 4, 2019, at 11:49 am, Adrian Perez de Castro <aperez at igalia.com> wrote:

Hello Aakash, WebKittens,

On Thu, 29 Aug 2019 18:01:56 -0400, Aakash Jain <aakash_jain at apple.com> wrote:

I want to update everyone with the further improvements I have made to new EWS. As always, please feel free to provide any feedback (either by filing bugs or contacting me directly).

New Features:

[...]

Thanks a ton for all the improvements in the EWS! Since we moved the GTK/EWS builders to the new queues they have been faster and more reliable. After solving a couple of minor hiccups right after the switch, the builders have not needed any manual intervention since (with the old system we would need to clean stray files from old builds now and then, but not anymore!)

Glad to hear. New EWS has been more reliable and easier to maintain at our end as well.

\o/

Only remaining queues on old EWS are windows, jsc and commit-queue, which I will be working on next.

Today Xan López and I talked a bit about the JSCOnly queue. We have a many builders targeting different architectures, which we would like to have moved to the new system.

In the case of the JSC port, instead of grouping all of the builders under a single “jsc” status bubble it would be better to have a bubble for each target architecture e.g. “jsc-armv7”, “jsc-arm64”, and so on.

Yes, my plan is to have separate bubbles for different architectures. I will touch base with you when I work on those queues.

Want more features?

Request early access to our private beta of readable email premium.