Subject: Dealing with incompatible changes in build system on LTS releases


Hi,

do we currently have a strategy on how to handle LTS releases in context of
incompatible changes  on the build system?

As far as I can see, the problem is (at least) twofold.

1. Incompatible changes on test-infra job definitions

There might be changes in our groovy files which make it impossible to
build/test an old branch on Jenkins. How do we intend to handle this? Of
course, in that cases we could run seed job and reset Jenkins to
corresponding old state but this will impact or even stall development on
master.

2. Incompatible changes on agents

Even worse, we might introduce changes on the agents itself, which will
even render it impossible to successfully seed to that legacy state. Do we
have any option to revert to an old Jenkins agent setup in such cases? I am
currently unaware of a link from apache repo to Jenkins configuration state
to enable restauration of (old) agents? Is there such thing?

Would it be possible/helpful to subdivide our Jenkins agent pool in some
way that seed job could be run only on a dedicated subgroup (which then
could be set to an old state)? If I recall correctly Yifan put a lot of
effort in migrating our agents to the newer jnlp approach.
 and used a 'private' agent to do require testing. I assume this has been a
manual setup and is not automated to be useful in such cases?

What do others think about this issue? Is it something to follow on or more
of a non issue?

Best,

michel