Firstly, you need a good contrast between the text and the background for readability.
Secondly, colour creates mood, and (e.g.) orange is often some kind of danger sign, subliminally arousing energy and excitement, perhaps, but also tension and anxiety.
One solution is to use your strong saffron for feature areas (e.g. header image) and a much lighter version of it (e.g. #f3d3af) as the background for your main text area. Another is to make your saffron even darker and the text very light, but I think it’s going to be a bit oppressive – still not conducive to the mood you want.
You might like to try some of the colour-design tools from The 28 best tools for choosing a colour scheme at Creative Bloq.