Thank you for your great and comprehensive responses.
I did that database clean-up and also connected the Redis with CockpitCMS. Seems that caching improved it a lot. But still, for an empty cache, it can be a problem.
For now, it solved my problems with Gatsby builds.
But still, I am really curious why it is happening and what is the real cause.
If I will find it, will update this thread.
Thank you for your help.