It is possible to just move from one kind of worldliness to another. Both reveal a life that neither sees or affirms a right view of God. That can be with or without religion.
Timothy Keller, in the introduction to his new commentary on Galatians, writes:
The Galatian Christians had been pagans, who were under the slavery of literal idolatry—’the basic principles of the world’ (4:3, 8-9). But here Paul once more makes his radical claim that pagan idolatry and biblical moralism (ie: keeping the laws of the Bible) are basically the same thing. The Galatians had been amoral liberals, and now they were about to become very moral conservatives.
Worldliness has many faces. The picture you have in mind of what worldliness looks like may or may not be accurate. One thing is true, however; at the core, it is a life that neither sees or affirms a right view of God.