# Uploading list if links rather than pasting them individually

**URL:** <https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749>\
**Category:** General\
**Tags:** custom-domain, pickaxe, knowledge-base\
**Created:** [July 27, 2024, 2:58pm UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749 "2024-07-27T14:58:31Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![kariim1177](https://avatars.discourse-cdn.com/v4/letter/k/f0a364/32.png) [@kariim1177](https://community.pickaxe.co/u/kariim1177)\
**Post date:** [July 27, 2024, 2:58pm UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749/1 "2024-07-27T14:58:31Z")

</div>

Hello i would like to know that can i upload list of links as txt files rather than pasting each link as i have a collection of 10000000 links to add to knowledgebase

---

<div class="post-metadata">

**Author:** ![mrzang](https://avatars.discourse-cdn.com/v4/letter/m/77aa72/32.png) [@mrzang](https://community.pickaxe.co/u/mrzang)\
**Post date:** [July 28, 2024, 9:30am UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749/2 "2024-07-28T09:30:03Z")

</div>

Plus one ! And better website side scrapping options to scape an entire domain for example @admin_mike

---

<div class="post-metadata">

**Author:** ![admin\_mike](https://yyz2.discourse-cdn.com/flex004/user_avatar/community.pickaxe.co/admin_mike/32/283_2.png) [@admin\_mike](https://community.pickaxe.co/u/admin_mike)\
**Post date:** [July 29, 2024, 7:16pm UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749/3 "2024-07-29T19:16:05Z")

</div>

Currently you can only add one website at a time. We are working on a better webscraping system.

It would be interesting to know what other options you would like to see with this feature.

---

<div class="post-metadata">

**Author:** ![Slabs](https://yyz2.discourse-cdn.com/flex004/user_avatar/community.pickaxe.co/slabs/32/381_2.png) [@Slabs](https://community.pickaxe.co/u/Slabs)\
**Post date:** [July 31, 2024, 1:18am UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749/4 "2024-07-31T01:18:39Z")

</div>

Same here, I think if we could add the sitemap.xml or upload a txt with all the urls would be a great start.

---

<div class="post-metadata">

**Author:** ![stephenasuncion](https://yyz2.discourse-cdn.com/flex004/user_avatar/community.pickaxe.co/stephenasuncion/32/3111_2.png) [@stephenasuncion](https://community.pickaxe.co/u/stephenasuncion)\
**Post date:** [July 31, 2024, 2:07am UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749/5 "2024-07-31T02:07:54Z")

</div>

Hi @Slabs,

You can upload `https://<domain>/sitemap.xml` into the knowledge base now.

---

<div class="post-metadata">

**Author:** ![Slabs](https://yyz2.discourse-cdn.com/flex004/user_avatar/community.pickaxe.co/slabs/32/381_2.png) [@Slabs](https://community.pickaxe.co/u/Slabs)\
**Post date:** [August 1, 2024, 3:44pm UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749/6 "2024-08-01T15:44:28Z")

</div>

Hi @stephenasuncion ,

Thanks for the response, I have tried adding [https://www.petsmart.ca/sitemap\_1.xml](https://www.petsmart.ca/sitemap_1.xml) and it did not work, is the tool only accepting urls ending on ‘/sitemap.xml’ ?

---

<div class="post-metadata">

**Author:** ![stephenasuncion](https://yyz2.discourse-cdn.com/flex004/user_avatar/community.pickaxe.co/stephenasuncion/32/3111_2.png) [@stephenasuncion](https://community.pickaxe.co/u/stephenasuncion)\
**Post date:** [August 1, 2024, 4:04pm UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749/7 "2024-08-01T16:04:17Z")

</div>

Hi @Slabs,

You should be able to add URLs ending with `sitemap_<index>.xml` or `sitemap.xml` now

---

<div class="post-metadata">

**Author:** ![Slabs](https://yyz2.discourse-cdn.com/flex004/user_avatar/community.pickaxe.co/slabs/32/381_2.png) [@Slabs](https://community.pickaxe.co/u/Slabs)\
**Post date:** [August 1, 2024, 11:29pm UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749/8 "2024-08-01T23:29:22Z")

</div>

Wonderful! Thanks for the fast update @stephenasuncion

---

<div class="post-metadata">

**Author:** ![skuan](https://yyz2.discourse-cdn.com/flex004/user_avatar/community.pickaxe.co/skuan/32/402_2.png) [@skuan](https://community.pickaxe.co/u/skuan)\
**Post date:** [August 1, 2024, 11:36pm UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749/9 "2024-08-01T23:36:37Z")

</div>

Hi @stephenasuncion @admin_mike  
I understand that you added a limit of 50 files per bot for the knowledge base and I appreciate your last updates to the KB setup, but why the URL is also limited to 50 links? Could we have a higher limit when it is only URLs? Depending on the case, 50 links is too little.

---

<div class="post-metadata">

**Author:** ![kariim1177](https://avatars.discourse-cdn.com/v4/letter/k/f0a364/32.png) [@kariim1177](https://community.pickaxe.co/u/kariim1177)\
**Post date:** [August 8, 2024, 7:34am UTC](https://community.pickaxe.co/t/uploading-list-if-links-rather-than-pasting-them-individually/749/10 "2024-08-08T07:34:16Z")

</div>

Hello Pickaxe team  
Just two three two three one things to know from you

1. are you planing for razorpay payment gateway as stripe is not yet available in India and I am not able to monitize my studio
2. do you have plans for gemini api as they are offering ai startup credits
3. can we have multiple options to use means knowledge base+ source llm or any plans for integration with pabbly?
