SillyTavern is one of the most popular interfaces to interact with LLMs. We have been working on developing an API and one of the first interfaces we wanted to integrate with was SillyTavern. We have done just that.
Requirements: Infermatic.ai Plus Tier subscription ($15/month)
Steps to integrate:
- After you subscribe to Infermatic.ai you can generate an API key. You will see a new option on the left-side menu bar called
API Keys. Select that and a modal will open to generate a new key or copy an existing key.
2. In the modal generate a new key or copy an existing key.
3. Run SillyTavern locally. To connect our API select the power socket icon and match each setting.
- API: Text Completion
- API: Infermatic
- Custom Endpoint (If selected vLLM or Aphrodite): https://api.totalgpt.ai
- Custom API Key: They key you copied from step 2 above.
Then hit ‘connect’ connect to our API. The available models drop down will populate with various models our API supports. Select one and use the same name listed in the Enter a Model ID section.
5. And that’s all, ready for you to enjoy it!
Now we are still learning so would love feedback on this integration. Feel free to join us in Discord to share your feedback.
