Hmm… Then again the Nightly release process failed. I always get excited, when crucial PR’s get merged, and then get disappointed, when release process fails.
This is something I don’t understand. I thought that you should have a stable, somewhat working code in order to run any tests, and to be able to know whether new PR’s are going to work or not. And then the release would be more like formality, just a way to put things out.
But now many of the tests done in the release phase are almost always failing. Are these tests different from the ones done in PR checks? And if they are catching some real bugs and code is not working, how can it be developed further? Or is so that the tests in release process reveal some problems, and those are addressed in the following PR’s.
I think there was some talk about this some time ago, but I couldn’t find it now.
So the difference is running over droplets vs on the same machines… the PR CI flow does not start droplets. Nightly (vetted code) does. It runs with more nodes and runs more “real” tests against them as a consequence.
Right now it’s our client-hammering test that appears to be tripping up release atm. Which is likely as we had to remove backpressure atm due to that impacting DKG and section startup (there’s a massive difference there).
Both of these things are being actively looked at. (and both may actually be coming from a client issue that’s cropped up w/r/t network knwledge, which handover has highlighted and I think the generation including DKG was exacerbating).
But we are getting there! (And i do so feel your frustration. You would not believe!)
Still curious, what is the difference between the cases of PR check and attempted merge? If I understand correctly, there is no change in code, but different tests fail in these two cases:
Amazing that it’s Thursday afternoon and nothing in this topic yet
And here I am sipping coffee [drums fingers]. I can’t wait to read the next installment of everything you wanted to know about building Safe Network. It’s been a gripping read so far.
I’ve seen a couple of interesting merges this week, at least one on consensus and I think there was one for DKG but I’m not sure they’ll make much difference to tests.
(stolen)