Yes that could be interesting.

But as we start with an offset (the given public key), can we still apply symmetry and endomorpshim optimization ?

Yes.

Let P be the public key (offset). Let be 'secret' the private key of P.

Let 's' be the start random private key you generate from the seed, and s*G = Q the current start point.

Instead of starting from P, you start from Q+P. Let's say Q' = Q + P

Now, you find a public key R' such that --> ripemd160(sha256(R')) = address with a certain prefix.

Your program works exactly like now, it finds only the private key of Q, not of Q'. Simply you get a partial key of Q instead of the final private key of Q'.

Now there are 3 possibilities:

if R' is a standard point (no symmetry / endo), then you have found R' = R + P and you know only the private key k of R (k*G = R), not the private key of R'. Then k is the correct partial private key.

The final user needs to add the k found to the secret private key of P (that he only knows) to get the final private key:

final private key = (k + secret) mod n (explanation: (k + secret)*G = k*G + secret*G = R + P = R')

if R' is a simmetric point, then R' = -(R + P). You found the private key k of -R (k*G = -R), nothing more.

The final user needs to build the final private key this way:

final private key = (k - secret) mod n (explanation: (k -secret)*G = k*G - secret*G = -R - P = R')

if R' is a endo point, then R' = lambda1*(R + P). (or lambda2)

You found the private key k of lambda1*R (k*G = lambda1*R), then:

final private key = ( k + lambda1*secret)) mod n

(explanation: (k + lambda1*secret)*G = k*G + lambda1*secret*G = lambda1*R + lambda1*P = R')