GEDCOM Crawler

Download your entire family tree

"197 GEDCOMs downloaded, 4832 to go...Ugh!"

Requirements

License

Known Limitations

• Only works with the LDS Ancestral File. I'm looking into making it work with RootsWeb too.
• An individual may have more information about them listed in the GEDCOM file for one family than another. GEDCOM Crawler neither merges the information nor attempts to determine which record is the most complete, but simply uses the first one it finds. I'll try to improve on that in a later version.

Support

Instructions

1. Download and unzip GEDCOM-Crawler.zip (By downloading GEDCOM Crawler, you agree to the above license terms). NOTE: The files use UNIX style linebreaks. When opening the files on other operating systems, ...things may look weird. I may make additional packages with different linebreak styles later.
2. Open gedcom-crawler-conf.pl in a text editor like Notepad, or better yet, a programmer's text editor.
3. If the computer on which GEDCOM Crawler will run has a host name (for example, if you are going to run it on your webserver), enter the host name in $myHostName. Otherwise you will need to find out the IP address of the computer and enter it in $myHostIP. (If you enter the host name, you may leave $hostIP blank.)
4. Save your changes (be sure to save as a plain text file).
5. If you are going to run GEDCOM Crawler on a different computer than the one you have it on, upload all four files to that computer.
6. If necessary, make all four files executable. For example, on a UNIX or Linux computer, you might type the following command on the command line from within the directory where the files are located: chmod 700 gedcom-crawler*
7. Find the family ID for the most recent ancestor on the line whose data you wish to download:
   1. Go to the Ancestral File Search Form
   2. Find the ancestor
   3. Go to their individual record
   4. Copy the "Family" link on the right side of the page across from their name
   (don't go to that page and copy the URL in your browser's address bar--that won't be the right one--instead, right click the "Family" link and select "copy URL" or something like that from that menu)
   5. The number after "familyid=" in that URL is the family ID.
8. From the directory where GEDCOM Crawler is installed, execute the following command, substituting in the desired family ID: ./gedcom-crawler.pl 123456 Here, the fun begins! GEDCOM Crawler will start downloading and combining GEDCOM files for your ancestors. Each time it downloads a file, it will tell you how many families' data it has processed, and how many family IDs it has found but not processed. Watch both numbers go up for a while as it finds more and more families! After the numbers, it will display one dot for each line in the GEDCOM file--some families are bigger than others.
   
   IMPORTANT: GEDCOM Crawler will create a file whose name is the family ID specified on the command line plus ".ged". If you already have a file by that name in the same directory as GEDCOM Crawler, it will be overwritten.
9. If you wish to download other family lines without getting duplicates, run GEDCOM Crawler again with the new family ID, adding a space and an "x" to the command line like this: ./gedcom-crawler.pl 567890 x This will prevent GEDCOM Crawler from clearing the family IDs it has already downloaded from the database.
10. If you have already cleared the family IDs from the database and wish to avoid duplicates, or already have a GEDCOM file containing Ancestral File family IDs which you don't wish to duplicate, run gedcom-crawler-skip-families.pl as follows to put the family IDs into the database (substituting in the name of your GEDCOM file): ./gedcom-crawler-skip-families.pl gedcom-file.ged
11. As with gedcom-crawler.pl, you may tell gedcom-crawler-skip-families.pl not to empty the database before it runs by adding a space and an "x" to the command line: ./gedcom-crawler-skip-families.pl gedcom-file.ged x
12. Send your GEDCOM files to your family members!