, 1658. Source:[ Wikipedia](https://en.wikipedia.org/wiki/File:Orbis_Terrarum_Nova_et_Accuratissima_Tabula_by_Nicolaes_Visscher,_1658.jpg)](img/map2.jpg)
Dmap is multi-application crawler that *reduces the complexity* in both executing Internet measurements and analyzing results.
Dmap uses a list domain names (scalable to many millions), and:
1. *Crawls* domains for DNS, HTTP, TLS, SMTP and a creates a screenshot of a webpage (if it's available)
1. *Converts* the raw measurement data and stores this into a 166-features unified SQL data model, so users can easily analyze the results using SQL.
We have been using Dmap since 2017 to analyze the .nl DNS zone.
Just run it and query it. More details on our [research paper](paper.pdf).
**IMPORTANT: data model changes (2018-08-20)**
* We keep improving Dmap continuously. As such, there are some changes from the version described on our [research paper](paper.pdf).
* More importantly, the [data model](datamodel) page in this website *refers to the model used in the [research paper](paper.pdf).*
**NEWS:**
* **2018-08-20**: We presented during the [IETF 102](https://www.ietf.org/how/meetings/102/) in Montreal, Canada.
* Full presentation at the [IEPG meeting](http://iepg.org/), on July 15,2018. Slides [available here](http://iepg.org/2018-07-15-ietf102/maura-iepg-102.pdf)
* Short presetation at the [Measurement and Analysis Reserach Work group (IRTF MAPRG)](https://datatracker.ietf.org/rg/maprg/about/). Slides [available here.](https://datatracker.ietf.org/meeting/102/materials/slides-102-maprg-dmap-automating-domain-name-ecosystem-measurements-and-applications-giovane-c-m-moura-00.pdf)
* **2018-06-06**: We will present our [Dmap paper](paper.pdf) at the [TMA 2018 conference](http://tma.ifip.org/2018/) on June 27th, 2018, in Vienna, Austria.
* **2018-06-05**: Website online
# [Demo dataset ](demo.html)
Very often, researchers spend an awful lot of time planning and executing measurements. The complexity of different data formats and issues that may emerge drain energy that could be otherwise be spent on research questions.
[Dmap](http://dmap.sidnlabs.nl) is intended to be an enabling platform and is designed to free researchers from the complexity of measurements and parsing complex data formats; rather, it automates the measurements and stores the data in an easy to use SQL database.
We have a Dmap database dump for the Alexa 1M domains list, which we cover in our [research paper](paper.pdf).
Want to give at try? Check the [demo dataset](demo.html) and the [data model](datamodel.html) for a description of the data model.
# [Research paper](paper.pdf)
For a full description of Dmap, please download our [research paper](paper.pdf)
```verbatim
Wullink, M., Moura, G.C. M., Hesselman, C.: Dmap:Automating Domain Name Ecosystem
Measurements and Applications. In: IFIP/IEEE Network Traffic Measurement
and Analysis Conference (TMA 2018). Vienna, Austria, 26-29 June 2018.
```
Bibtex:
```verbatim
@inproceedings{wullink2018-dmap,
author = {Wullink, Maarten and Moura, Giovane C. M.
and Hesselman, Cristian},
title = {{Dmap: Automating Domain Name Ecosystem Measurements and
Applications}},
booktitle = {IFIP/IEEE Network
Traffic Measurement and Analysis Conference (TMA 2018)},
year = {2018},
month=jun,
address = {Vienna,Austria}
}
```
# Contact
* E-mail: `sidnlabs at sidn dot nl`
* Last update: Tue Aug 21 09:05:44 UTC 2018
Copyright (C) 2018 SIDN Labs