Llama2 at home with Ancient P40s
Contents
Every one wants a personal AI assistant at home, but running high quality LLMs (those with 70 billion parameters or more) today requires GPU hardware that is out of reach for most.