What are rules of thumb comparing feasibility ysis vs just starting a project?

How does one determine whether it is most cost effective to just starting a project/investment and let the results be the feasibility ysis (so-to-speak) as opposed to the time, effort, investment necessary to do a feasibility study ahead of time before going ahead with the project/investment?