When debugging a web app and tracking down problems, especially newly created and deployed apps that aren't working, sometimes all that's needed is to look and see what's deployed to help determine the issue. This is trivial to do using the App Service Editor available from the Azure Portal.
Once logged into the portal, navigate to the website or specific deployment slot of the app where you'd like to browse the files. Viewing the menu of options on the left-hand side, select App Service Editor:
This will redirect to a location in a new browser tab which will show the folder structure and physical files present. Within the utility there are other useful functions available along the left-hand side in addition to the logging output to help diagnose and have insight into your deployed site.
This feature is still in 'Preview' mode so expect some changes over time to the utility, possibly being renamed, and on occasion being down. As noted you can always FTP into the directory as well as a secondary option.