Maps
Depending on what you want to do for your mapping needs the cost could vary from the cost of some plain butcher paper to large erasable tabletop maps to even some expensive online mapping tools. This cost will usually hit the DM, but usually they are nice resources to have. When I started DM'ing I bought a smaller erasable tabletop map and some washable markers so I could draw out the map as we went along. It worked great because all the pieces were visible, I could have some liberties on designing the board a little, but the issue always came in losing time when I needed to draw a new map. Over a little time I tried finding other alternatives short of buying additional maps that I could lay out over the others and eventually I found Roll20.net, a site that provides a ton of online resources for maps and even has built-in technology that integrates character sheets, music (from SoundCloud), and chat features. What's better is the cost is $0 for basic functionality, but if you want some more bells and whistles you can donate to the team and they provide different features like dynamic lighting and line of sight features among many more. Currently I run all my campaigns through Roll20 regardless of online or in-person. The in-person campaign had a minor cost of $15 for an HDMI cord to connect my computer to an HDTV so everyone can see well.
Books
Arguably this is where the largest cost can be for any D&D start-up. Books when they are new usually run about $50 and of course with each new edition that means buying nearly $200 of books for the Dungeon Masters Guide (DMG), Players Handbook (PHB), Monster Manual 1 (MM1), and perhaps the new Dungeon Master Screen. The cost only increases if you prefer to run pre-build campaigns as those books can cost between $20-50 depending on the length. This cost is tricky to deal with and has been one of the factors that have delayed my endeavor into the newer editions.
I like having the books because I really enjoy them, but they aren't fully required. Currently there are a lot of online resources that contain a lot of the information contained within the PHB and the MM1 (and a few that have more than MM1), but it isn't always the easiest to find (which is why I have many of them compiled on the Resources Page). Many of the sites I have listed have information straight from the books and even the item generators use similar randomizing tables from the books to generate the items.
The best way to save on books (provided you want to buy them) is really only buying the books you know you will want and if you want to save a little you can buy PDF versions of the books at a pretty good discount from DND Classics. If you prefer to have a physical copy of the book, I found that Amazon had some pretty decent deals for the books and occasionally I can find one at discount bookstores.
Ultimately, it is up to you if you want to invest in getting the books. There are plenty of online resources that can help you with rules, items, classes, spells, etc. where getting the books aren't vital. I would say that as long as your DM is pretty well versed in the rules you shouldn't need to have the books, but you may need the players to be on the ball with their notes so they know exactly what every feat, ability, and spell does. It may be a little more work, but it will save you some money in the long run.
Tokens
I've been playing D&D for about 5.5 years now and I have never used any D&D miniatures to play. When I started playing, we used a random dice to represent our character, eventually we upgraded up to Lego figurines. When I started DM'ing about 4.5 years ago I used wooden checkers with numbers written on them and chess pieces to represent the characters. Any of the tokens that I've used have always been things I found around the house that could be a good representative of a character. While I understand the charm of having a figurine that represents your character and has been painted to look exactly as you want your character to look, I never had the manual dexterity to do the work as well as I'd like to, plus I didn't really want to invest in them. Just as with the books, this really is a personal preference whether you want to have tokens or not.
Despite that I've listed this as a cost, depending on the party's M.O. you may have sessions where you will not need tokens. I have run one campaign where all we used was dice and role-play to play through a few sessions. Yes, this did mean substantially less combat, but the party I was running was more of a diplomatic envoy at that time and were more into solving the problem without resorting to war. So as the situation varies you may not need fancy tokens, but realistically you can choose whatever you want to be your tokens, be they free, cheap, or premium.
Dice
Dice are the one thing that you really can't start a campaign without. Since the game is completely based on rolling d20s it is important to have at least one if not more. This doesn't mean you have to go online and buy a set as a player (though it is nice to have your own) and it doesn't mean as a DM you have to buy a pound of dice to provide to your players (though it is nice to have them available in case someone forgets theirs or in case someone needs more than one of a type of dice). There are many apps and websites that have dice rollers that functionally are the same as rolling dice (though some argue it isn't perfectly random, but that's another topic). Most of these apps are free and you don't have to buy the app if you are only looking to have it roll dice. If you insist on having your own set of dice, the best way to save is to get a simple one-color set of dice as those tend to be cheaper as a set. Dice that are two-tone, speckled, metal, etc. tend to have a higher production cost so it will cost a little more to get them.
Altogether, while starting a new edition of D&D can be pricey, especially since there aren't enough online resources to support a book-free play to start, it is definitely possible to cut costs of playing D&D a substantial amount to a point where you can start getting things slowly as you start to enjoy the game more. Once you dive into the game, maybe you find that you're looking up info from a certain book more that you'd prefer to have a copy that you can put bookmarks into. Perhaps as a DM you'd prefer to use a computer monitor as the map so you can cut the cost of tokens and maps by playing online (this also allows you to meet from the comfort of your home on nights that it may not have otherwise been convenient to meet.
In theory, it is possible to play D&D for whatever price you are comfortable playing, regardless if that value is free or more the resources are out there to get you playing.