I learned using a web server, but most good web developers have their own installation from what I've read. It's just the way I like to do things. Even when I've had my own Apache server I never really used it for much testing.
MySQL is accessible by various server-side programming languages, including Python and PHP. I prefer PHP because it's pretty easy, essentially made for web development, interacts with SQL really simply, and has just a ton of support. But if you're really comfortable with Python you could go with that. PHP wouldn't be that hard to learn, though. I suppose more knowledgeable programmers (i'm not really that familiar with Python) might say that it depends on what kind of app you were trying to make.
If all you need to do is learn mysql, you neither need php nor python. All you need is mysql. Just install mysql server. It comes with a command line client. Type your sql commands in the mysql prompt. play around.
Learning how to use mysql is pointless though if you can't make scripts that interact with the database's. I prefer php to python especially since the free hosting sites out there more often support php than python. Also, in regards to setting up a webserver on your local machine, I use IIS, the default web server that comes with windows. It is possible to install php on it, and it saves the headache of running apache and what not.
I know IIS sucks, but its great for local testing in my opinion, and if you ever want to learn asp.net, iis is nice/necessary to have.
In conclusion, learn php and learn it by making a website that has something to do with data in a database, thats probably the best way to do it. And also learn asp.net while you are at it because you can get a really nice paying job.