Could it be that the pre existing cottage was built using lime mortar? This would have the effect of increasing the alkalinity of your soil to a point above the tolerance level for earthworms (between 5.0 and 8.0 is optimal) - normal neutrality of soil is 7, so if there was/is a high level of residual lime mortar in the soil it will not be suitable for earthworm activity. Have you done a Ph test? If you have a lot of success raising brassicas (i.e. no clubroot) that might indicate high Ph levels.
Just some of my observations
If you need to gather a herd of earthworms to be a resident population I will post my fishing pal's method of collecting them
Just some of my observations

If you need to gather a herd of earthworms to be a resident population I will post my fishing pal's method of collecting them

Comment