Map Tile Downloader

This is a Python script generated by the Gemini large language model, designed to download map tiles from CartoDB services for specified geographic regions and zoom levels. It supports downloading by either geographic latitude/longitude bounds or direct tile XY coordinates, and offers advanced features like multi-threading, resume functionality, retries, custom User-Agents, and proxy support.

Features

• Flexible Tile Sources: Supports various preset CartoDB map styles (e.g., dark_all, light_all, voyager, etc.) and allows for custom tile URL templates.
• Two Download Modes:
  • Geographic Bounds Download: Download tiles for a specific area by specifying minimum/maximum latitude and longitude. The program automatically calculates and downloads all tiles covering that region.
  • Tile Coordinate Download: Directly specify the X and Y tile coordinate ranges for download, useful when you already know the tile indices.
• Multi-Zoom Level Support: Capable of downloading all tiles within a specified min_zoom to max_zoom range in one go.
• Resume Capability: Skips downloading tile files if they already exist, allowing you to resume interrupted downloads easily.
• Efficient Concurrency: Utilizes multi-threading (ThreadPoolExecutor) to speed up tile downloads.
• Error Handling & Retries: Supports a retry mechanism for failed downloads and specifically handles common HTTP errors like 400 Bad Request, 404 Not Found, and 403 Forbidden to prevent futile retries.
• Custom Request Headers: Allows setting a custom User-Agent to mimic browser access.
• Proxy Support: Configurable HTTP/HTTPS proxy settings for downloads.
• Progress Display: Uses the tqdm library to show download progress for each zoom level and overall.
• Global Map Download Optimization: Intelligently handles latitude inputs like -90.0 to 90.0 to correctly identify and download all tiles for a full global map.
• Logging: Detailed log output for easy tracking of the download process and troubleshooting.

Usage

1. Setup

Ensure you have Python 3 installed on your system. Then, install the required dependencies:

pip install requests tqdm

It's recommended to install these dependencies within a Python virtual environment to avoid cluttering your global environment.

2. Running the Script

The script is configured via command-line arguments. Here are some common usage examples:

Downloading Global Tiles from Z=0 to Z=5 for dark_all Style

python enhanced_tile_downloader.py ^
    --url dark_all ^
    --min_zoom 0 ^
    --max_zoom 5 ^
    --min_lat -90.0 ^
    --min_lon -180.0 ^
    --max_lat 90.0 ^
    --max_lon 180.0 ^
    --output "Global_Tiles_Dark_Z0-5" ^
    --workers 10 ^
    --retries 3

Parameter Explanation:

• --url: Specifies the tile type (e.g., dark_all) or a complete custom tile URL template.
• --min_zoom, --max_zoom: Sets the starting and ending zoom levels for download.
• --min_lat, --min_lon, --max_lat, --max_lon: Defines the geographic bounds of the download area. Note that when downloading global maps, inputs like -90.0 and 90.0 for latitude are correctly handled by the program.
• --output: The root directory where tiles will be saved. The program will create it if it doesn't exist.
• --workers: The number of concurrent download threads (default is 10).
• --retries: The number of times to retry a failed tile download (default is 3).

Downloading Tiles for a Specific Region (e.g., Downtown Shanghai)

python enhanced_tile_downloader.py ^
    --url voyager ^
    --min_zoom 10 ^
    --max_zoom 14 ^
    --min_lat 31.20 ^
    --min_lon 121.45 ^
    --max_lat 31.25 ^
    --max_lon 121.50 ^
    --output "Shanghai_Voyager_Z10-14" ^
    --workers 15

Downloading by Tile XY Range

python enhanced_tile_downloader.py ^
    --url light_all ^
    --min_zoom 12 ^
    --max_zoom 12 ^
    --min_x 3500 ^
    --max_x 3510 ^
    --min_y 1600 ^
    --max_y 1610 ^
    --output "Custom_Tiles_Z12"

Using Proxies and Custom User-Agent

python enhanced_tile_downloader.py ^
    --url dark_all ^
    --min_zoom 0 ^
    --max_zoom 1 ^
    --min_lat 0 ^
    --min_lon 0 ^
    --max_lat 10 ^
    --max_lon 10 ^
    --output "Proxy_Test" ^
    --http_proxy "http://user:pass@your_proxy_host:port" ^
    --https_proxy "https://user:pass@your_proxy_host:port" ^
    --user_agent "MyCustomAgent/1.0"

3. Checking Downloaded Tiles

After the download is complete, all tiles will be organized in the structure: {output_directory}/{z}/{x}/{y}.png.

For example, for the Global_Tiles_Dark_Z0-5 directory:

Global_Tiles_Dark_Z0-5/
├── 0/
│   └── 0/
│       └── 0.png
├── 1/
│   ├── 0/
│   │   ├── 0.png
│   │   └── 1.png
│   └── 1/
│       ├── 0.png
│       └── 1.png
└── ...

---

Important Notes

• Legal Use: Please ensure your download activities comply with the terms of service of the tile provider (e.g., CartoDB). Excessive or unauthorized downloading may lead to IP blocking.
• Network Connection: Tile downloading requires a stable internet connection and sufficient bandwidth.
• Storage Space: High-zoom level tiles can be very numerous. Ensure you have ample disk space. For instance, global tiles from Z=0 to Z=18 can consume terabytes of storage.
• Floating-Point Precision: Latitude and longitude parameters are floating-point numbers. While the program has optimizations, very minor precision issues might still occur in extreme edge cases.

---

Dockerfile for the Tile Downloader

How to use this Dockerfile

1. Create requirements.txt: First, create a file named requirements.txt in the same directory as your enhanced_tile_downloader.py and Dockerfile. This file should list the Python dependencies your script needs:

   requests
   tqdm
   

2. Build the Docker Image: Open your terminal or command prompt, navigate to the directory where you saved the Dockerfile, enhanced_tile_downloader.py, and requirements.txt, and run the following command:

   docker build -t tile-downloader .

   This command builds a Docker image named tile-downloader based on your Dockerfile. The . at the end indicates that the build context is the current directory.

3. Run the Docker Container: Once the image is built, you can run your tile downloader inside a Docker container. You'll need to specify the download arguments and also mount a volume so that the downloaded tiles are saved to your host machine (otherwise, they'd just be inside the container and disappear when the container stops).

   Here's an example command to download global tiles from Z0-Z5, saving them to a local folder named my_downloaded_tiles:

   docker run --rm -v "$(pwd)/my_downloaded_tiles:/app/Global_Tiles_Dark_Z0-5" tile-downloader ^
       --url dark_all ^
       --min_zoom 0 ^
       --max_zoom 5 ^
       --min_lat -90.0 ^
       --min_lon -180.0 ^
       --max_lat 90.0 ^
       --max_lon 180.0 ^
       --output "Global_Tiles_Dark_Z0-5" ^
       --workers 10 ^
       --retries 3

   • --rm: This option automatically removes the container once it exits.
   • -v "$(pwd)/my_downloaded_tiles:/app/Global_Tiles_Dark_Z0-5": This is the volume mount.
     • "$(pwd)/my_downloaded_tiles": This specifies a directory on your host machine (the current directory pwd followed by my_downloaded_tiles).
     • :/app/Global_Tiles_Dark_Z0-5: This specifies the corresponding directory inside the container where the script will try to save the tiles (based on your --output argument). Make sure the path after /app/ matches your --output argument's value. If you change your --output folder, you'll need to change this part of the volume mount too.
   • tile-downloader: This is the name of the Docker image you built.
   • The remaining arguments (--url, --min_zoom, etc.) are passed directly to your Python script running inside the container.

   Note for Windows users: If you're using Command Prompt, you might need to change $(pwd) to %cd% and adjust path separators if necessary, or use Git Bash/WSL for a more Unix-like experience. For PowerShell, $(pwd) should work.

This Dockerfile provides a solid foundation for packaging and running your tile downloader.

---

Thank you for using this tool! Feel free to provide feedback or suggestions.