Script Example: Move and Unzip File; Rename Zipped File

coffee mug and notepad on desk

I wanted to share a script I developed as part of a regional GIS group Cumberland County is a member of.  Each member county extracts their data and uploads it to an FTP site.  Then a extract-transform-load process (several scripts) is completed to update the regional datasets.  The simple utility script I’m sharing accomplishes two goals:

  1. Copies and unzips each county’s geodatabase in the extract directory on the server
  2. Renames the zipped folders on the FTP directory of the server to include a date stamp (mm-dd-yyyy)

The whole process is simplified by using a for in loop.  There is a loop for the copy/extract process and for the zipped file re-naming process.  Each loop cycles through a list of county names which correspond to directories on the server (i.e., a directory named CumberlandCounty).  The script utilizes the zipfile and shutil modules to move files among directories and unzip the zipped geodatabases.

Rename zipped file with current date


# The following modules are utilized by the script
import os, sys, time, datetime, traceback, string, zipfile, shutil

# Get current date and time
currentTime = datetime.datetime.now()
# format current date and time to "04182017" for example
dateToday = currentTime.strftime("%m%d%Y")

# list of county names
countyList = ['County1', 'County2', 'County3']

# root directory of FTP site
ftpParentDirectory = r'path\to\ftp root directory'

# Loop through each county
for county in countyList:
   # name of County Directory Part
   countyDirectoryPart = county + 'County'
   # directory to move into
   newDirectory = os.path.join(ftpParentDirectory, countyDirectoryPart)
   # zipped folder
   zippedFolder = county + '.zip'
   # zipped folder with date stamp
   zippedFolderDate = '{}_{}.zip'.format(county, dateToday)
   # move into county directory
   os.chdir(newDirectory)
   # get list of files in directory
   filesInDirectory = os.listdir(newDirectory)

   # if CountyName.zip file exists in directory rename it
   if zippedFolder in filesInDirectory:
         # copy and rename file
         shutil.copyfile(zippedFolder, zippedFolderDate)
         # write message
         message('Copied file {} and renamed it to {}'.format(zippedFolder, zippedFolderDate))
   else:
         message('The file {} does not exist.  Could not copy and rename file'.format(zippedFolder))
   # end if
# end loop

Move zipped folder to extract directory and unzip


# Base directory for data extraction
extractBaseDirectory = r'path\to\base\extract\directory'

# Loop through each county
for county in countyList:
     # name of County Directory Part
     countyDirectoryPart = county + 'County'
     # directory to move into
     newDirectory = os.path.join(ftpParentDirectory, countyDirectoryPart)
     # zipped folder
     zippedFolder = county + '.zip'
     # directory of zip file on ftproot
     zipDirectory = os.path.join (newDirectory, zippedFolder)
     # move into county directory
     os.chdir(newDirectory)
     # directory for county extract
     countyExtractDirectory = os.path.join(extractBaseDirectory, countyDirectoryPart)

     # get list of files in FTP directory
     filesInDirectory = os.listdir(newDirectory)
     # if CountyName.zip file exists in directory, copy into extract directory and unzip
     if zippedFolder in filesInDirectory:
         # Copy County.zip file into extract directory
         shutil.copy2(zippedFolder, countyExtractDirectory)
         message('Copied file {} to directory {}'.format(zippedFolder, countyExtractDirectory))
         # Unzip file
         copiedZipFile = os.path.join(countyExtractDirectory, zippedFolder)
         with zipfile.ZipFile(copiedZipFile, "r") as z:
             z.extractall(countyExtractDirectory)
         message('Unzipped file {} in directory {}'.format(zippedFolder, countyExtractDirectory))
         # delete County.zip file in county extract directory
         os.remove(copiedZipFile)
         message('Removed file {} from directory {}'.format(zippedFolder, countyExtractDirectory))
         # delete County.zip file in county ftp directory
         os.remove(zipDirectory)
         message('Removed file {} from directory {}'.format(zippedFolder, newDirectory))
     else:
         message('The file {} does not exist.  Could not copy file to {}'.format(zippedFolder, countyExtractDirectory))
     # end if
# end for loop

 

As I am writing about this script, I am realizing I probably could create some helper functions and have a more concise script, as there is some repeated code.  Still working on DRY!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s