Hi,
Most of the time the older chicks are ready to leave the nest first. But usually the rest will follow in a few days. This is when you remove the nest. I don’t know what type of lining it is, but they are probably just picking at it. If they are really eating it, this can get impacted and kill them. I wouldn’t recommend using a nest with any type of cloth liner. Finches usually like the woven nests that look like a hut. A liner is just something that can get dirty and possibly get them sick. If you breed your birds again in a few months, I’d get a nest without a liner.
Thank you for asking Lafeber,
Brenda