ENGM 204: Data Analytics Project Lab

Public Data Sets on AWS provides a centralized repository of public data sets that can be seamlessly integrated into AWS cloud-based applications.

Open, Secure, Social and Linked data--building the most meaningful, collaborative, and abundant data resource in the world.

Dataset of 200,000 true and false claims

Important, commonly-used datasets in high quality, easy-to-use & open form as data packages

Search for datasets and related data spread across multiple data repositories on the web.

Open datasets on everything from government, health, and science to popular games and dating trends.

Datasets gathered by Meta (Facebook).

The following list of data sources has been collected and categorized for your convenience. The list has been limited to those for which there is a reasonably simple process for importing csv files. Most of the data sets listed below are free, however, some are not.
If an (R!) appears after source this means that the data are already in R format or there exist R commands for directly importing the data from R.

Data behind the Inside Airbnb site is sourced from publicly available information from the Airbnb site. The data has been analyzed, cleansed and aggregated where appropriate to faciliate public discussion.

Open Trade Statistics is an independent project that values reproducible research and provides tidy trade data. They are heavily inspired by R community values and Open Licenses views on freedom. They focus on commodities data and focus on data processing and reproducibility instead of data visualization.

Datasets from the ANT (Analysis of Network Traffic) Lab, whose goal is to goal is to improve the Internet by discovering new ways to understand network topology, traffic, use and abuse.

Community Resource for Archiving Wireless Data At Dartmouth. CRAWDAD began in 2004 at Dartmouth College as a place to share wireless network data with the research community. Its purpose was to enable access to data from real networks and real mobile users at a time when collecting such data was challenging and expensive. The archive has continued to grow since its inception, and starting in summer 2022 is being housed on IEEE DataPort.

Financial information from across Africa.

Open data repository with data from across the African continent.

Over 1.6 million EU public sector datasets from 36 countries

UK collection of economic, population and social research data.

Detailed global data on trade statistics by the UN

Statistical tables and regional profiles by country.

Allows you to retrieve vintage versions of economic data that were available on specific dates in history.

The purpose of Data.gov is to increase public access to high value, machine readable datasets generated by the Executive Branch of the Federal Government.

Various types of publications that are primarily statistical in nature, including books, magazine series, and true statistical releases.

Online database consisting of more than 148,000 economic data time series from 59 national, international, public, and private sources. FRED, created and maintained by Research Department at the Federal Reserve Bank of St. Louis.

Housing and mortgage data.

US Census links and explanations and mapping software.

Access and analyze health data from 26 cities, for 34 health indicators, and across six demographic indicators.

Data on how health and disease are passed on between generations--not just by genes, but also through social, personal, and environmental surroundings.

Medicare data to provide information and analysis about national, regional, and local markets, as well as hospitals and their affiliated physicians.

Largest collection of longitudinal hospital care data in the United States.

Includes clinical care provider quality information, nationwide health service provider directories, databases of the latest medical and scientific knowledge, consumer product data, community health performance information, government spending data.

Detailed mortality and population data

Mammographic Image Analysis Society (MIAS) database and the Digital Database for Screening Mammography (DDSM)

Information about services and procedures provided to Medicare beneficiaries by physicians and other healthcare professionals, with information about utilization, payment, and submitted charges organized by National Provider Identifier (NPI), Healthcare Common Procedure Coding System (HCPCS) code, and place of service.

The Cancer Data Access System ("CDAS") is a website where you may request data recorded from various research studies. For some studies, you may also request images or biospecimens.
CDAS provides extensive public documentation for each study, including a trial summary, an overview of the data collected, and a searchable database of research projects and publications.

Data visualization, searchable statistics, and interactive queries on health and health care.

Sharing neuroimaging data

This data set contains two sets of pictures of household objects, created by the RoboFEI@Home team to develop object detection systems for a domestic robot.
The first data set was created with objects from a local supermarket. Product brands are typical from Brazil. The second data set is composed of objects from the RoboCup@Home 2018 OPL competition.

The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images.

HMDB is collected from various sources, mostly from movies, and a small proportion from public databases such as the Prelinger archive, YouTube and Google videos. The dataset contains 6849 clips divided into 51 action categories, each containing a minimum of 101 clips.

ImageNet is an image database organized according to the WordNet hierarchy (currently only the nouns), in which each node of the hierarchy is depicted by hundreds and thousands of images. The project has been instrumental in advancing computer vision and deep learning research. The data is available for free to researchers for non-commercial use.

Indoor scene recognition is a challenging open problem in high level vision. Most scene recognition models that work well for outdoor scenes perform poorly in the indoor domain. The main difficulty is that while some indoor scenes (e.g. corridors) can be well characterized by global spatial properties, others (e.g., bookstores) are better characterized by the objects they contain.

The corpus contains more than one billion words of text (25+ million words each year 1990-2019) from eight genres: spoken, fiction, popular magazines, newspapers, academic texts, and (with the update in March 2020): TV and Movies subtitles, blogs, and other web pages. Can be downloaded.

The Movies Corpus contains 200 million words of data in more than 25,000 movies from the 1930s to the current time. The Movie Corpus serves as a great resource to look at very informal language -- at least as well as with corpora of actual spoken English. In addition, the Movies Corpus is much larger than any other corpus of informal English

WordNet® is a large lexical database of English. Nouns, verbs, adjectives and adverbs are grouped into sets of cognitive synonyms (synsets), each expressing a distinct concept. Synsets are interlinked by means of conceptual-semantic and lexical relations. The resulting network of meaningfully related words and concepts can be navigated with the browser. WordNet is also freely and publicly available for download. WordNet's structure makes it a useful tool for computational linguistics and natural language processing.

A data dump of the Free Music Archive (FMA), an interactive library of high-quality, legal audio downloads.

Data and code behind some of its articles and graphics.

Maintains a data archive of more than 250,000 files of research in the social and behavioral sciences. It hosts 21 specialized collections of data in education, aging, criminal justice, substance abuse, terrorism, and other fields.

Explore 100,000 HD video sequences of over 1,100-hour driving experience across many different times in the day, weather conditions, and driving scenarios. Video sequences also include GPS locations, IMU data, and timestamps.

Browse, visualize, & download curated public transport network data for 20+ cities.
Data formats: GTFS, network edge lists, event lists, GeoJson, SQLite databases.