Generative AI tools like Google Gemini and ChatGPT are probably making their way into most of the tech we use every day. Large Language Models, or LLMs, are what these tools are built on. LLMs are networks that have been taught on huge amounts of human data so that they can produce realistic text, images, and videos.
But you don't need a cloud app to use these LLMs. You can run them on your own computer as well. You can use these models even when you're not online, and you don't have to give Google or OpenAI your questions and chats.
With the power of its RTX 30 and RTX 40 line graphics cards, Nvidia has now released Chat with RTX, its own local LLM app. You can put a creative AI robot on your computer and change it to fit your needs if you have one of these GPUs.
What to do first
Make sure you have the most up-to-date drivers for your Nvidia GPU before you begin. The GeForce Experience app on your PC can help you with this. Next, go to the Chat with RTX download page. Windows 11 and at least 16GB of RAM are required to run the tool. You also need a GeForce RTX 30/40 series GPU or an RTX Ampere or Ada Generation GPU with at least 8GB of VRAM.Please keep in mind that Nvidia calls this a "demo app," which we think means it's not fully finished yet, so you may encounter some bugs and other issues. There are also a lot of LLMs in this download, making it about 35GB in size. This is something to keep in mind if you don't have a lot of disk room or internet speed. It also takes a while for the software tool to finish everything it needs to do.
The Chat with RTX application ought to eventually appear on your Start menu. When you run it, your usual web browser will open with the software interface after a few moments of processing. You can choose from Llama and Mistral, two open-source AI models, up in the upper left corner. After it is finished, you may begin sending instructions just like you would with Google Gemini or ChatGPT.
You will be familiar with the capabilities of these generative AI engines if you have ever used an LLM. Get assistance with texts, emails, documents, and more. Get complicated subjects explained. Ask questions that you may ordinarily look up on the internet (such as "what's a good party game for four adults?").
This AI bot follows normal guidelines, so try to be as specific and detailed as you can, and be cautious about placing too much trust in the responses you receive (especially since this is a "demo"). Chat with RTX will constantly attempt to provide you with an answer based on the vast amount of internet material that it has been trained on, but it is not really appropriate for giving answers that need to be current because it cannot seek up current information on the web.
There are three buttons located at the bottom of the interface: delete chat, which will erase your conversation history and allow you to start over; undo, which will take you back to the previous prompt; and generate a new response from the last prompt, if you're not too happy with the current one. Currently, the only method for exporting responses is to copy and paste the content.
Including YouTube videos and your own data
Chat with RTX offers a few helpful features, even in this early version. One of them is the capacity to base its responses on documents you supply: Perhaps a week's worth of study, a stack of reports that require analysis, or even the entirety of the fanfiction you've been penning. Choose Folder Path under the Dataset title, then point the application in the direction of the folder containing the documents you wish to use.After the program has scanned the folder you've sent it to which could take a minute or two if the folder contains a lot of files you can begin entering your search terms. After searching the text for pertinent answers, the bot will identify the file or files it used by naming them at the conclusion of the message. You can ask the bot to create new text depending on the text you feed it, check facts, or request summaries.
To emphasize, you shouldn't currently base boardroom decisions on this technology, as it is still in its early stages and is known to be less than 100% accurate. Still, it's entertaining to experiment with an LLM that can work from papers you provide it, be they volumes of poetry or transcripts of interviews.
In relation to transcripts, Chat with RTX may also evaluate YouTube videos and provide comments based on the transcripts that are attached to the clips. (Our testing indicates that it is capable of producing transcripts for videos without any existing ones.) You can even use this to have the application play through a complete playlist on YouTube at the same time with several clips.
After choosing YouTube URL as the dataset, enter the video or playlist's URL in the field below. In the box on the right, you can indicate how many videos from the playlist you wish to add if you're working with one. Once you've done that, click the download icon on the far right of Chat with RTX. The transcript text will then download and be processed, ready for any further instructions you may receive.
Similar to the document-scanning feature, this can be helpful for extracting summaries or highlighting key material, but we discovered that it was the most difficult aspect of the application to utilize (which, to be fair, is still in version 0.2). Chat with RTX can be a helpful tool to try if you need quick answers to lengthy movies and playlists that you don't have time to watch, even though the app occasionally confused which video we were referring to.
0 Comments