HN Jobs

IPUMS | ipums.org | Data Engineer | Minneapolis, MN | Onsite | Full-Time IPUMS is a leader in the field of quantitative social science research and the largest disseminator of census and demographic data to the world’s academic research community. Or, to put it another way - we’re on a mission to gather, process, link and publish billions of records spanning hundreds of years and more than 100 countries to demographers, historians, economists, environmental scientists, journalists, policymakers, and others around the globe, who then use the data to do amazing research and make the world a better place. We are currently seeking a software developer to join our data linking team that is working on large scale data manipulation and creating highly performant distributed software systems. In this role, you will be developing software that links records representing the same individual and their family relationships across every US Census from 1850 - 1940. There are no consistent identifiers across historical censuses, so sophisticated matching algorithms will be required. You will be building on research and IT effort from the past decade to create a dataset that, once released, will enable critical new avenues of research for demographers and historians. This work will also lead to additional data linking opportunities for datasets from around the world and throughout history. You will be working in close collaboration with expert historians, demographers, and data scientists. We use Apache Spark (PySpark and SparkSQL) for a majority of our data processing, with C++ in critical areas. This is an excellent opportunity to grow your career with a cutting -edge software shop at the University of Minnesota and contribute to work that has impact around the globe. Required Skills: BA/BS degree required. Two years of work experience in the areas of application/web/systems development with a related BA/BS degree, or four years of work experience in the areas of application/web/systems development with a non-related BA/BS degree. This role requires technical experience and proficiency with: - Experience developing software for high performance and/or distributed computing - Experience with databases (relational or NoSQL) - Large dataset manipulation - Linux/UNIX operating systems, including command-line See https://www.ipums.org/data-engineer for more details and instructions on how to apply.