I’m not a “doomsday prepper “, but I recognize the need for having access to a Local Large Language Model (LLM) “AI” in emergency off-grid situations where you have no power and no network connectivity, and maybe you need access to medical, or survival knowledge, or whatever information a local AI could provide. So that’s kind of why I took on this project.
-
Is capable of running at least one AI Large Language Model (LLM) or multiple LLMs at an acceptable generation speed (preferably 2+ tokens per second)
-
Requires absolutely no connectivity (after initial provisioning of course)
-
Is handheld, extremely portable, and ruggedized if possible
-
Accepts multiple power sources (Solar, hand-crank, AC/DC, etc) and provides multiple output types
-
Has a camera, microphone, speaker, and touch screen for input
-
Doesn’t require any separate cords or power adapters that aren’t already attached / included in the box itself
Those were the basic requirements I made before I began my research. Originally, I had planned to do the whole thing using a Raspberry Pi with an AI accelerator, but the more I researched, I realized that an android-mini tablet or a budget unlocked android phone would probably be the best and easiest option. It’s really the perfect form factor and can readily run LLMs, so why reinvent the wheel when I could just use a cheap mini android tablet.
The second part of the solution was that I wanted multiple power sources with a small form factor that closely matched the Android tablet / phone form factor. After an exhaustive search, I found a Solar + Hand Crank + USB Lithium battery power bank that had some really unique features. The one I selected for this project has a solar panel, and a hand crank for charging, it included 3 built-in cords for power output, 2 USB types for power input, it even had a bonus flashlight and is ruggedized and waterproof.
Thanks to selection of feature-complete parts involved, there is very little actual assembly required to complete this whole project (other than securing the tablet to power source / generator and installing and downloading the software required).
- XREAL Beam Pro 8GB RAM / 256 GB - $249 on Amazon
You basically need an android phone minus the phone. The XREAL Beam Pro that I chose has Wi-Fi, Bluetooth, and a good bit of storage for multiple LLMs. The system RAM is sufficient enough to run at least a Q4 quantized 4 billion parameter AI Large Language Model (LLM). What makes the XREAL Beam Pro android device so special for this use case is that it has 2 USB C ports (I’m not aware of another tablet / phone that has that). The reason this device has 2 ports is because it was originally made to connect to a pair of XR/AR glasses so you could watch media, play games, use the glasses as a virtual monitor, etc. So, while one port charges the device (from our special power source that we’ll discuss in a minute), the other port can output to a monitor or presumably connect to a docking station so you could run the whole thing in a desktop computer configuration with a mouse and keyboard (if you wanted to do that). This second port offers a lot of configuration flexibility while the other port charges the whole setup. It also removes the need to keep up with stupid pass-through charging dongles (which can get lost easily).
So yes, you can use whatever cheap tablet or old phone you have access to, but this little XREAL Beam Pro offers some compelling features for our AI-in-a-box use case. Oh yeah, it also has a 3D stereoscopic camera built in for when you need off-grid spatial photography for whatever reason.
Note: there are two other versions of the XREAL Beam Pro. There’s one that’s $50 cheaper than the one I selected, but it only has 6GB RAM / 128 MB storage. I thought it was a bad idea to skimp on resources for this use case, so I spent the extra $50 for double the storage and 2 more GB of system RAM. There is also a model that is $50 more than the one I bought. It adds a 5G cell radio for cellular reception (which may also give the device GPS location services, but I have not verified this because I don’t own the upgraded model). I went with the middle model with the extra RAM and storage but no 5G.
Here are the three XREAL Beam Pro options: (again, I don’t recommend the cheapest one because I feel like I need as much system RAM as possible for acceptable AI inference speed when running the AI LLM):
-
X-REAL Beam Pro 6GB RAM /128 GB Storage (Wi-Fi / Bluetooth) - $199 – Not the model that I recommend due to it only having 6GB of system RAM. It may work, but its answers may be slow. I would spend the extra $50 to move up to the next model.
-
X-REAL Beam Pro 8GB RAM /256 GB Storage (Wi-Fi / Bluetooth) - $249 – Recommended model if you don’t plan on needing 5G connectivity. Adds 2 GB of system RAM to the base model (for a total of 8GB) and has 256 GB of storage (vs. 128 GB on the lower mode) which allows you to store a lot of LLMs and other applications / data, books, movies, etc.
-
X-REAL Beam Pro 6GB RAM /256 GB Storage (Wi-Fi / Bluetooth / 5G) - $299 – Best choice from a connectivity options perspective, but also the most expensive choice. Adds the option of having 5G cell network connectivity which requires a SIM and cell service provider to work. May possibly also add GPS capability which could be useful in survival situations, but I have not confirmed that this function exists on this model.
2. The AI-in-a-box Server Power Source (Solar Generator / Hand-crank Generator / Battery / USB power)
They shoved a lot of technology into this little box. At its core, it’s basically a traditional USB power bank but this one charges the internal battery in multiple ways. It’s got a solar panel on the back that appears to actively charge the battery as a second power generation source that charges it regardless of what your primary charging source is doing. It also has a built-in hand crank generator for when the sun isn’t shining, flashlight, USB C, USB A, and lightning cables for power output that stow away when not in use. In addition to the solar and hand crank it has USB A and USB C for charging its internal battery as well.
There are many other versions of this type of device, but this particular one had everything I needed. In case you can’t find that particular unit, here are a few other similar options. Most of them range from $45 to $60.
To make this whole thing into a convenient hand-carriable unit, you will need to affix the power source to the AI server, I recommend using a super strong yet easily removable Velcro-like material from 3M. You will need to purchase:
- 3M Command 15 lb Large Picture Hanging Strips- $9 on Amazon
The 3m picture hanging strips are a special kind of super strong Velcro-like adhesive that in my opinion holds way better than traditional Velcro. But essentially any sticky-backed Velcro you can find will work. I swear by the 3m stuff though. I use this material to strap the tablet to the power bank. It holds well.
Other than the Android OS (XREAL’s specific flavor is called Nebula OS) all you need is:
This is a free Google-developed AI inference app that is optimized for hosting LLMs on mobile devices and allows camera access for vision-capable AI models. It doesn’t use GGUF model format but rather something called .TASK format. There are plenty of lightweight AI models available in the .TASK format including
-
and tons more on HF and KAGGLE
Here is Google’s full Google AI Edge Gallery: User Guide to help you get started with installing the software and the AI models.
For the ultra-serious preppers who are worried about electronics getting fried from ElectroMagnetic Pulses (EMPs) from nuclear blasts or whatever, you could try and mitigate that risk by storing this whole setup in a Faraday Cage Bag. Will it protect it from an EMP? I have no idea and hope I never have to find out, but in theory it should.
- SenseCAP Card Tracker T1000-E - $59 - Amazon
This Meshtastic compatible tracker adds GPS capabilities and low-power LORA text messaging (assuming you are near other LORA / Meshtastic devices). If you had a couple of these Meshtastic devices, you could essentially track other members of your survival party and text message them using the Meshtastic mesh network. These little trackers have a pretty good range and being able to text your fellow party members over long distances in an emergency situation when all other forms of telecommunications are down could prove very valuable.
I haven’t added this yet, but it is on my roadmap of things to add. LORA / Meshtastic is a mostly community-built network and seems like maybe it would be accessible when the big telecom companies are down for whatever reason.
If you want a big screen monitor that you can carry in your bugout backpack, these glasses effectively put a giant virtual monitor in front of your face. A cool but expensive extra, but not a must-have for this use case. The XREAL Beam Pro was built to provide a really cool spatial computing interface for these XR glasses, so it makes sense to use them if you’re into that kind of thing. I’ve linked the cheapest available ones I could find above, but there are other more state-of-the-art ones currently available (also expensive). Here is the high-end version of the glasses:
I might try and add some kind of AI Retrieval Augmented Generation (RAG) features but am not sure how performative they would be given that the hardware is already pushing the envelope by just running an LLM. It may be better to fine tune a model using RAFT rather than trying and to add RAG.
I definitely need to add a ruggedized case to my Beam Pro. There weren’t many options available when I first bought it, but it appears that there are now.
Ensure that you provision the tablet device that you are using to host the LLMs. Make sure you have run all system and firmware updates and loaded all Android patches to get to the latest version of the software. The beam pro features the Nebula OS, which contains special software specific to the Beam Pro’s spatial computing abilities (for use with the optional XR glasses
2 - Install and configure Google AI Edge Gallery
This software is the core part of the AI-in-a-box Server. It provide the Large Language Model inference server capabilities and the chat interface that you will use to communicate with the LLM that you will download and run offline on the SERVE-AI-VAL Box. Download and install the app
3 - Download AT LEAST a 4 billion (4b) parameter AI Local Large Language Model (LLM) from the Google AI Edge Gallery
The “brain” of your AI-in-a-box is called a Large Language Model or LLM. You can find these models in the Google AI Edge Gallery app and also on repositories such as Hugging Face These models come in various sizes known as parameter counts. Due to the memory and CPUI limitations of portable devices such as the X-REAL Beam Pro that we are using as our hardware, we must choose an LLM with a relatively low parameter count so it will fit on the device and perform at a reasonable speed.
Some of the available LLMs are text-only while some feature vision capabilities which allow you to submit a picture to the LLM in a chat and have it interpret the picture for you. This vision capability could be useful in survival situations for emergency medical or orienteering scenarios when the LLM might be able to provide better answers if it has a picture to go along with the situation you are describing for it. Follow these instructions for obtaining an LLM to install on the XREAL Beam Pro (or other capable android-based tablet).
You’ll want to test out the LLMs you download to ensure that they are not only working, but can also answer your questions at a reasonable speed. “Reasonable speed” for this type of device would be about 2 to 3 tokens per second. This may seems slow, but just remember that it is a miracle that these cheap low power, low resource devices can run LLMs at all, so 2-3 tokens given the available resources is great.
The true test of the SERVE-AI-VAL Box is when you have no network connectivity available. You want to ensure that the AI is capable of answering questions without a network connection. Turn on airplane mode, or turn off both Wi-Fi and Bluetooth on the X-REAL Beam Pro’s settings to ensure that you can simulate not having a connection. Once you have done this, have another chat with the AI model in Google AI Edge Gallery to ensure that it is responding when all connectivity is turned off.
- Using two strip of the 3M picture hanging strips that you purchased, cut the pull tab release ends of the strips off as shown below:
- and secure the sticky adhesive side of two of the strips to either the case of the X-REAL Beam Pro, or the X-REAL Beam Pro itself (if you don’t have it in a case or don’t want a case). I recommend adhering the strips to the following locations of the back of the X-REAL Beam Pro (see picture below)
- Take two more of the 3M picture hanging strips and repeat the procedure above, but do this on the back side of the MQOUNY Solar Charger Power Bank. Be sure not to cover any of the areas where the charging wires are located. (see picture below for best location of the adhesive strips.
-
Ensure that you have pressed each trip into place for at least a 30 seconds each so that the adhesive can bond with the case and the X-REAL Beam Pro. IMPORTANT: Let the adhesive set for at least 30 minutes before you attempt to connect the pieces together.
-
Release the USB C charge cable from the back of the MQOUNY Solar Charger Power Bank before going to the next step, make sure it is out of the way before moving to step 6.
- Line the 3M strips on the back of the X-REAL Beam Pro and the corresponding strips upi attached to the MQOUNY Solar Charger Power Bank and press with enough force until you hear the adhesive “snap” together. Note, the 3M material is not like traditional Velcro, it has a much more solid hold, but won’t hold at all until it is “snapped” together. Be careful not to apply too much pressure on the screen when connecting the X-REAL to the Solar charger.
- Connect the MQOUNY Solar Charger Power Bank’s USB C charge cable to the left hand port at the bottom of the X-REAL Beam Pro. This is the charging port, the other port WILL NOT Charge the device. The other port is strictly for connecting X-REAL glasses (or a monitor that accepts USB C input). The charge port is the one that does not have the glasses icon next to it.
Congratulations, you now have a fully functional Solar / Hand-crank / Battery / USB-chargeable AI-in-a-box that you can use in emergency situations.
You will probably also want to install other apps, books, and other materials that may be handy in emergency situations. I recommend ensuring that these apps DO NOT require any kind of active internet connection to function so you can ensure that they work when you have no connectivity available. You might want to download:
-
First Aid apps or books
-
Survival guides
-
A PDF reader
-
Offline maps
-
Measurement and conversion apps / charts
-
Flashlight and signaling apps
Again, you’ll need to make sure that whatever you download can work without an available internet connection. Turn on Airplane mode to test your apps.
Note: Some of the links above are affiliate links.