Japan is a very cash-oriented economy. While most shopping and retail accept card without problems, simpler stores and tourist spots usually accept cash only.
Thanks to the safety in the country, there is absolutely no danger in carrying large amounts of cash with you. But, how much do you need, and how easy it is to withdraw money from ATM's?
As a general rule of thumb, the best rates for exchange are withdrawing cash from Japanese ATM's on the go, but one should pay attention to withdraw fees by the banks - some offer free withdraws, others have fees.
Exchanging your currency for Yen in Japan is usually best in Exchange Offices in Japanese airports - do not use your country's Airport Exchange Offices as they offer one of the worst rates (because it is usually dictated by availability of Yen).
Exchanging currency with your bank or Exchange Offices before travel is also not considered a good option, though it allows you to exchange in increments months before the trip.
Cash or Card?
Hotels, large malls and stores and department stores will accept cards, but smaller street vendors, traditional locations and the ticket booth at most Shrines, Temples and Parks might be cash only - for that, it is extremely important to keep some cash with you.
Estimate Budget Tool
will also estimate how much cash you should carry.
Considering that the normal entrance fees vary between ¥100 and ¥1.000, and that snacks will also hardly be more pricey, carrying a good deal of ¥10, ¥50, ¥100 and ¥500 coins is the best way around, specially because you will be using a lot of automated vending machines (most accept ¥1.000 paper bills too). You should have around ¥2.000 ~ ¥5.000 depending on your destination and needs in cash per day to guarantee no problems - get used with those coins!
You will find ATM's that accept international Credit/Debit cards on most convenience stores and sometimes major stations. ATM's at 7-Eleven combinis and Post Offices (and some Aeon Bank ATM's inside Aeon Malls) are usually the go-to to withdraw cash.
Before leaving, contact your bank to make sure your card is enabled for international withdraw, and be sure to explicitly express your need for that - most banks will block your card if it is used in another country without prior warning. Also, just for your own safety, if you are not sure your card is ready, check if you have the international tool number for your bank in case you need to call them.
Important notice: regardless of the method you are used to use in your card, be sure to know your main password or PIN, since international ATM's do not use any other method like pass-keys.
Most banks will charge a fee for international withdraw, while some don't. A known debit card that have no fee is Charles Schwab Bank
Other than zero-fee debit cards, the best rate is often at Haneda Airport or Narita Airport exchanges (links are to exchange houses at the airport, check them!). Some banks might also offer near market-level exchange rates, though you might want to check the fee for foreign withdraw and check if it is cheap or not. Finally, as a rule of thumb, the worst rates are at your local country exchange houses.