Search Engines accepting submissions
Past Efforts to Allow Public Contributions to Search Engines
Some search engines—especially open-source or decentralized ones—have experimented with allowing users to contribute directly to their data or indexing pipeline. These efforts generally fall into a few categories:
Open Indexing Initiatives
- YaCy (Decentralized)
- Fully P2P search engine where users run nodes that crawl the web.
- Public participation is core to the architecture.
- Contributions: Run a YaCy node, add seed URLs, and share index.
- Common Crawl
- Massive open dataset of web crawl data.
- Not a search engine itself, but enables researchers and projects to build on shared data.
- Contributions: You can’t directly submit URLs, but can process and build tools from the dataset.
- Mojeek
- Independent crawler-based engine.
- No public URL submission interface, but accepts emails for site inclusion.
- Past experiments included crawling niche topic indexes.
- SearX/SearXNG
- These are metasearch engines, but forks have included the ability to add custom engines or prioritized domains.
- IndexNow (Microsoft, Yandex)
- A push protocol for site owners to notify search engines of new/updated content.
- Supported by Bing and others; anyone can implement it.
- Contributions: Add
indexnow.jsonto your site and ping endpoint when content updates.
Search Engines that Accept URL Submissions or Additions
| Search Engine | Public URL Submission | Notes |
|---|---|---|
| ❌ (No longer available) | Removed general URL submit tool in 2020 | |
| Bing | ✅ Submit URL tool | Can submit via Bing Webmaster Tools or IndexNow |
| Brave Search | ❌ | Index is private; no public submission |
| Mojeek | ✅ Submit page | Manual submission form available |
| Yandex | ✅ | Through Yandex.Webmaster interface |
| DuckDuckGo | ❌ | No public crawler; relies on Bing and other APIs |
| Swisscows | ❌ | Index closed; proprietary |
| Presearch | ❌ | Distributed search interface; not an indexer |
| Gigablast | ✅ | Formerly accepted submissions; status unclear as project was largely retired |
| Qwant | ❌ | Based on Bing API and own index; no submission interface |
If You’re Building Your Own Engine
If your goal is to make your own engine accept public submissions:
- Add a simple “Submit a URL” form with a CAPTCHA
- Store URLs in a queue to be fetched by your crawler
- Use robots.txt to respect exclusion
- Optionally allow CSV/JSON sitemap uploads or APIs for power users
