Public Project Meeting - February 25, 2021

Continuing the series of bi-weekly public project meetings, here is the timeline of the meeting held on February 25, 2021,5:30 PM CET. (Below times are on IST)

Meetings are held on the #selenium-tlc channel on Selenium Slack.

Diego Molina 9:54 PM

@titusfortner @barancev @harsha509 @luke @jimevans @simonstewart

Luke Hill 9:54 PM

Yip. I’m here if needed.

Diego Molina 9:55 PM

The only topic I have today, to make it a short meeting, is: “what is needed for beta 2” Feel free to add/propose any other topics

Luke Hill 9:59 PM

Alongside that, do we have a rough idea of what would enable us to release a 4.0.0 proper (i.e. what milestones we need to hit)

Diego Molina 10:01 PM

Let’s wait a few minutes to see if we have enough people to start the meeting

Titus Fortner 10:04 PM

I have a couple minor things I want to do before beta2, but the sooner the better for me :)

Simon Stewart 10:04 PM

I’m in a planning meeting, but I’ll follow along slowly

Titus Fortner 10:05 PM

(Element#dom_attribute is the main one I want to finish). I think Java, C# both have that one already

Diego Molina 10:05 PM

Ok, then let’s start with the ones who are around. Only one topic so far: “what is needed for beta 2”

What comes to my head: fixing the leak :slightly_smiling_face:

Simon Stewart 10:05 PM

Crush the leaks. Ensure CDP on Grid to a Docker instance works

Diego Molina 10:06 PM

Regarding the leak, after reading more about the AsyncHttpClient issues and their google group…

10:07

they mention that each AsyncHttpClient instance has its own pool

Diego Molina 10:07 PM

which is why they recommend to have one single instance of it

Puja Jagani

After you pointed out the potential problem area, I attempted to use single instance of the AsyncHttpClient today and still saw the leak :confused: Will try and dig deeper tomorrow. Let me know if you want me to try something in particular.

Diego Molina

One single instance for the whole distributor?

Puja Jagani

One single instance as we had earlier, similar to https://github.com/SeleniumHQ/selenium/blob/selenium-4.0.0-alpha-7/java/client/src/org/openqa/selenium/remote/http/netty/NettyClient.java#L41 java/client/src/org/openqa/selenium/remote/http/netty/NettyClient.java:41 private static final AsyncHttpClient httpClient = https://github.com/SeleniumHQ/selenium|SeleniumHQ/seleniumSeleniumHQ/selenium | Added by GitHub

Puja Jagani

This was just to narrow down the problem area. It was something I was trying to eliminate if creating a new instance is the problem or the way we handle the response (blocking bit you mentioned earlier)

Puja Jagani

Distributor heap with single instance :see_no_evil:

Diego Molina 10:08 PM

additionally you can limit the size of the pool

so I am planning to tweak the client configuration and see if that helps

10:09

sadly, the docs for AsyncHttpClient are inexistent, but it seems like a robust implementation it is used for Gatling

10:09

(a loat test tool)

10:10

load*

David Burns 10:12 PM

@simonstewart what’s left, assuming its everything, to “Ensure CDP on Grid to a Docker instance works”

Simon Stewart 10:12 PM

(In a meeting)

Titus Fortner 10:13 PM

each of the bindings need to be able to implement that part (getting debugger address from caps, etc)

David Burns 10:13 PM

@titusfortner it’s done?

10:14

it gets it from se:options

Titus Fortner 10:14 PM

I don’t think he’s done yet with the first part

he’s changing that

Simon Stewart 10:14 PM

se:cdp now I updated the bindings that use it

David Burns 10:14 PM

@simonstewart touched my code… filthy java person in my python code

Simon Stewart 10:14 PM

The “CDP in Grid” stuff will add se:cdpVersion

Luke Hill 10:14 PM

One of the new things I noticed in ruby (Not sure if it’s relevant across the board). Was the new chrome cdp stuff needs to be able to work in local/remote instances.

Simon Stewart 10:14 PM

“My precious….”

David Burns 10:14 PM

:stuck_out_tongue:

Simon Stewart 10:15 PM

@luke that’s what the se:cdp capability allows

Luke Hill 10:15 PM

One step ahead of me :smile:

Simon Stewart 10:15 PM

The Selenium Server knows how to forward CDP traffic

Titus Fortner 10:15 PM

yeah, Ruby code has never actually allowed users to benefit from browser specific functionality in Remote WebDriver because of subclassing blah blah

Luke Hill 10:15 PM

So check back again in beta2 is the answer basically.

Titus Fortner 10:15 PM

I have a PR to address it

Simon Stewart 10:15 PM

I’m going to make the local drivers also set se:cdp

Titus Fortner 10:16 PM

With that PR, the Ruby CDP code works with server only if it is on localhost, so we’ll also have to update to what Simon has done

Luke Hill 10:16 PM

We “know” what we need to do. Which is half the battle.

Titus Fortner 10:17 PM

With @p0deje been gone for a while and @twalpole being super busy it’s just a matter of bandwidth. :)

well, partly, I only know “ish”

Diego Molina 10:17 PM

it’s the chance for @luke to do some commits :slightly_smiling_face:

Luke Hill 10:18 PM

I can try help where possible. But I’m not quite on their standard.

Titus Fortner 10:18 PM

I’d also like to figure out how to do a different gem publishing for CDP versions

10:19

it’s more important that we be able to update the CDP versions to match the browser versions than Selenium methods to CDP methods, so I’d like to be able to release the artifacts independently, and let users toggle it somehow.

Luke Hill 10:19 PM

By Easter I will hopefully be in somewhere more stable again either renting or buying my first place. So yeh should be able to do more in evenings. This is going to be my fifth property move in just over 2 years

Titus Fortner 10:22 PM

How much work is there still in the leak investigation? (I ask because I literally have no idea what all is involved)

Diego Molina 10:23 PM

Not sure, I’ve invested this week and at least I’ve pinned it down to the Distributor

10:24

It eats memory over time when new tests are executed, and never returns it

I am now checking if the problem is really related to the AsyncHttpClient

10:25

well, more related to the way we use it

David Burns 10:25 PM

@diemol @Puja Jagani knows this is a priority so feel free to rope her in :slightly_smiling_face:

Diego Molina 10:26 PM

they recommend to have a single instance, and in the Distributor we have one instance per registered node, and one instance to do the health checks, and another one that I have not pinned down to see what it is

Simon Stewart 10:30 PM

Because each instance needs its own config

10:31

If we could change config per request, we’d be fine

Diego Molina 10:31 PM

they have a different base url, right?

Simon Stewart 10:31 PM

Right

10:32

And possibly time outs

Anything on the config object

Diego Molina 10:33 PM

I did not see different timeouts, but we will need them when we allow users to configure the timeout in Grid

Puja Jagani

I think the RequestBuilder allows to set timeouts per request.

Diego Molina 10:34 PM

so, hm, interesting… now sure what to do now :slightly_smiling_face:

Simon Stewart 10:35 PM

Once again, I shall mutter about writing our own http client based on netty, starting from the one we have for domain sockets