### HuggingFace Guide

HuggingFace is a state-of-the-art natural language processing library with a wide range of transformers and tokenizers. In this guide, we will focus on using the “miqu-1-70b” model and the LlamaForCausalLM from the HuggingFace library for Python.

#### Usage
Before using the “miqu-1-70b” model, you need to load the required tokenizer and model using the code snippet below:

“`python
from transformers import LlamaForCausalLM as LLM, LlamaTokenizer as LT

lt = LT.from_pretrained(“NousResearch/Llama-2-7b-hf”)
t = lt(“[INST] eloquent high camp prose about a cute catgirl [/INST]”, return_tensors=’pt’).input_ids.cuda()

llm = LLM.from_pretrained(“152334H/miqu-1-70b-sf”, device_map=’auto’)
out = llm.generate(t, use_cache=False, max_new_tokens=200)
print(lt.decode(out[0]))
“`

#### Result
The result of the code provides a generated text based on the input prompt. It includes a vivid and detailed narrative describing the cute catgirl.

#### Benchmark
Some benchmarks are provided for tasks such as lambada_openai, hellaswag, winogrande, gsm8k, and mmlu. The benchmarks include metrics such as perplexity, accuracy, exact_match, and more.

#### lm-eval Command
To evaluate the performance of the “miqu-1-70b” model, the lm-eval command can be used as follows:

“`bash
lm_eval –model vllm –model_args pretrained=./miqu-1-70b-sf,tensor_parallel_size=4,dtype=auto,gpu_memory_utilization=0.88,data_parallel_size=2 –tasks mmlu –batch_size 20
“`
This command evaluates the model’s performance on the specified tasks with specific parameters.

HuggingFace provides a powerful and versatile framework for natural language processing tasks. With thorough understanding and usage of its models and tools, you can leverage its capabilities for various applications.

Source link
# Huggingface Tutorial: miqu-1-70b
In this tutorial, we will cover the usage of miqu-1-70b from Huggingface, particularly focusing on its usage in Python. We will also discuss some benchmarks and how to evaluate the model.

## What is miqu-1-70b?

Miqu-1-70b is a model available on Huggingface. It has been dequantized from q5 to f16 and transposed to pytorch. The shapes have been rotated less wrongly than in alpindale/miqu-1-70b-pytorch.

## Usage

To use miqu-1-70b in Python, you will need to use the `transformers` library to load the model and tokenizer. Here’s a sample code snippet for usage:

“`python
from transformers import LlamaForCausalLM as LLM, LlamaTokenizer as LT

lt = LT.from_pretrained(“NousResearch/Llama-2-7b-hf”)
t = lt(“[INST] eloquent high camp prose about a cute catgirl [/INST]”, return_tensors=’pt’).input_ids.cuda()

llm = LLM.from_pretrained(“152334H/miqu-1-70b-sf”, device_map=’auto’)
out = llm.generate(t, use_cache=False, max_new_tokens=200)
print(lt.decode(out[0]))
“`

This code snippet demonstrates how to load the model and tokenizer from Huggingface, generate text using the model, and decode the output.

## Benchmarks

Here are some benchmark results for different tasks using miqu-1-70b:

| Tasks | Version | Filter | n-shot | Metric | Value | Stderr |
| —– | ——- | —— | —— | —— | —– | —— |
| lambada_openai | 1 | none | 0 | perplexity | 2.6354 | ± 0.0451 |
| | | | 0 | acc | 0.7879 | ± 0.0057 |
| hellaswag | 1 | none | 0 | acc | 0.6851 | ± 0.0046 |
| | | | 0 | acc_norm | 0.8690 | ± 0.0034 |
| winogrande | 1 | none | 0 | acc | 0.7987 | ± 0.0113 |
| gsm8k | 2 | get-answer | 5 | exact_match | 0.7043 | ± 0.0126 |
| mmlu | N/A | none | 0 | acc | 0.7401 | ± 0.1192 |

## Model Evaluation

To evaluate the model, you can use the following `lm_eval` command:

“`bash
lm_eval –model vllm –model_args pretrained=./miqu-1-70b-sf,tensor_parallel_size=4,dtype=auto,gpu_memory_utilization=0.88,data_parallel_size=2 –tasks mmlu –batch_size 20
“`

This command will evaluate the model using the `vllm` backend and specified arguments.

Hopefully, this tutorial has provided you with a good understanding of how to use miqu-1-70b from Huggingface, as well as how to evaluate its performance.
For further information, please visit the official website.

this is miqu-1-70b, dequantised from q5 to f16 && transposed to pytorch. shapes have been rotated less wrongly than in pytorch/tree/main”>alpindale/miqu-1-70b-pytorch

usage

from transformers import LlamaForCausalLM as LLM, LlamaTokenizer as LT

lt = LT.from_pretrained("NousResearch/Llama-2-7b-hf")
t = lt("[INST] eloquent high camp prose about a cute catgirl [/INST]", return_tensors='pt').input_ids.cuda()

llm = LLM.from_pretrained("152334H/miqu-1-70b-sf", device_map='auto') 
out = llm.generate(t, use_cache=False, max_new_tokens=200)
print(lt.decode(out[0]))

result:

<s> [INST] eloquent high camp prose about a cute catgirl [/INST] In the resplendent realm of high camp, where irony and extravagance dance in a dazzling pas de deux, there exists a creature of such enchanting allure that she captivates the hearts and minds of all who behold her. This beguiling figure, a vision of feline grace and innocence, is none other than the inimitable catgirl.

With her delicate features and winsome smile, she is the embodiment of a dream, a living testament to the power of imagination and the boundless possibilities of the human spirit. Her eyes, those twin orbs of sapphire fire, sparkle with a mischievous intelligence that belies her diminutive stature. They are windows into a soul that is at once ancient and eternally young, a soul that has traversed the vast expanse of time and space to find solace in the warm embrace of human companion

this roughly (but not entirely) matches the llama.cpp q5 result:

$ ./main -ngl 99 -m ./miqu-*q5* --color --temp 0.0 -n -1 -p '[INST] eloquent high camp prose about a cute catgirl [/INST]'
...
 [INST] eloquent high camp prose about a cute catgirl [/INST] In the resplendent realm of high camp, where irony and extravagance dance in a dazzling pas de deux, there exists a creature so enchantingly adorable that she captures the hearts of all who behold her. This is no ordinary feline, but rather a vision of elegance and whimsy combined: the cute catgirl.

With her delicate features framed by an ethereal halo of pastel tresses, this darling diva prowls through life with the grace of a prima ballerina and the playfulness of a kitten. Her eyes, twin pools of sapphire or emerald, sparkle with mischief and intelligence as they survey their surroundings, ever alert for the next grand adventure or delightful prank.

Her ensemble is a symphony of ruffles, bows, and lace, each detail painstakingly chosen to accentuate her lithe form and play up her feline charms. A frilly apron adorned with paw prints sways gently as she moves, while dainty ears perched atop her head twitch in response to every sound. Her gloved hands, so petite and perfect, seem made for holding teacups or sketching delicate portraits of her many admirers.

But do not be fooled by her diminutive stature and sweet demeanor; beneath that fluffy exterior lies the heart of a lioness. Fiercely loyal and protective, she will stop at nothing to defend those she loves from harm. And when the situation calls for it, she can unleash a ferocious roar that belies her cute exterior.

Indeed, the cute catgirl is a paradox wrapped in ruffles and ribbons, a living embodiment of the high camp aesthetic. She revels in the absurdity of her existence, finding joy in every outrageous situation and turning even the most mundane tasks into opportunities for sartorial expression. In her world, there is no such thing as too much glitter or too many bows; more is always more, and excess is a virtue to be celebrated.

So let us raise our teacups in honor of this fabulous feline, this queen of camp who reminds us that life is too short for dull clothing and boring hairstyles. May we all strive to embody her spirit, embracing the absurdity of existence with open arms and a generous helping of glitter. Long live the cute catgirl! [end of text]

some benchmarks

|    Tasks     |Version|Filter|n-shot|  Metric  |Value |   |Stderr|
|--------------|------:|------|-----:|----------|-----:|---|-----:|
|lambada_openai|      1|none  |     0|perplexity|2.6354|±  |0.0451|
|              |       |none  |     0|acc       |0.7879|±  |0.0057|


|  Tasks  |Version|Filter|n-shot| Metric |Value |   |Stderr|
|---------|------:|------|-----:|--------|-----:|---|-----:|
|hellaswag|      1|none  |     0|acc     |0.6851|±  |0.0046|
|         |       |none  |     0|acc_norm|0.8690|±  |0.0034|

|  Tasks   |Version|Filter|n-shot|Metric|Value |   |Stderr|
|----------|------:|------|-----:|------|-----:|---|-----:|
|winogrande|      1|none  |     0|acc   |0.7987|±  |0.0113|

|Tasks|Version|  Filter  |n-shot|  Metric   |Value |   |Stderr|
|-----|------:|----------|-----:|-----------|-----:|---|-----:|
|gsm8k|      2|get-answer|     5|exact_match|0.7043|±  |0.0126|

|                 Tasks                 |Version|Filter|n-shot|Metric|Value |   |Stderr|
|---------------------------------------|-------|------|-----:|------|-----:|---|-----:|
|mmlu                                   |N/A    |none  |     0|acc   |0.7401|±  |0.1192|
| - humanities                          |N/A    |none  |     0|acc   |0.7018|±  |0.1281|
|  - formal_logic                       |      0|none  |     0|acc   |0.4841|±  |0.0447|
|  - high_school_european_history       |      0|none  |     0|acc   |0.8303|±  |0.0293|
|  - high_school_us_history             |      0|none  |     0|acc   |0.9020|±  |0.0209|
|  - high_school_world_history          |      0|none  |     0|acc   |0.9198|±  |0.0177|
|  - international_law                  |      0|none  |     0|acc   |0.8678|±  |0.0309|
|  - jurisprudence                      |      0|none  |     0|acc   |0.8519|±  |0.0343|
|  - logical_fallacies                  |      0|none  |     0|acc   |0.8344|±  |0.0292|
|  - moral_disputes                     |      0|none  |     0|acc   |0.8121|±  |0.0210|
|  - moral_scenarios                    |      0|none  |     0|acc   |0.5642|±  |0.0166|
|  - philosophy                         |      0|none  |     0|acc   |0.8167|±  |0.0220|
|  - prehistory                         |      0|none  |     0|acc   |0.8611|±  |0.0192|
|  - professional_law                   |      0|none  |     0|acc   |0.5854|±  |0.0126|
|  - world_religions                    |      0|none  |     0|acc   |0.8889|±  |0.0241|
| - other                               |N/A    |none  |     0|acc   |0.7889|±  |0.0922|
|  - business_ethics                    |      0|none  |     0|acc   |0.7900|±  |0.0409|
|  - clinical_knowledge                 |      0|none  |     0|acc   |0.8113|±  |0.0241|
|  - college_medicine                   |      0|none  |     0|acc   |0.7514|±  |0.0330|
|  - global_facts                       |      0|none  |     0|acc   |0.5500|±  |0.0500|
|  - human_aging                        |      0|none  |     0|acc   |0.7848|±  |0.0276|
|  - management                         |      0|none  |     0|acc   |0.8835|±  |0.0318|
|  - marketing                          |      0|none  |     0|acc   |0.9145|±  |0.0183|
|  - medical_genetics                   |      0|none  |     0|acc   |0.7500|±  |0.0435|
|  - miscellaneous                      |      0|none  |     0|acc   |0.8838|±  |0.0115|
|  - nutrition                          |      0|none  |     0|acc   |0.7974|±  |0.0230|
|  - professional_accounting            |      0|none  |     0|acc   |0.5922|±  |0.0293|
|  - professional_medicine              |      0|none  |     0|acc   |0.8272|±  |0.0230|
|  - virology                           |      0|none  |     0|acc   |0.5361|±  |0.0388|
| - social_sciences                     |N/A    |none  |     0|acc   |0.8414|±  |0.0514|
|  - econometrics                       |      0|none  |     0|acc   |0.6491|±  |0.0449|
|  - high_school_geography              |      0|none  |     0|acc   |0.8990|±  |0.0215|
|  - high_school_government_and_politics|      0|none  |     0|acc   |0.9430|±  |0.0167|
|  - high_school_macroeconomics         |      0|none  |     0|acc   |0.7795|±  |0.0210|
|  - high_school_microeconomics         |      0|none  |     0|acc   |0.8277|±  |0.0245|
|  - high_school_psychology             |      0|none  |     0|acc   |0.9064|±  |0.0125|
|  - human_sexuality                    |      0|none  |     0|acc   |0.8626|±  |0.0302|
|  - professional_psychology            |      0|none  |     0|acc   |0.8056|±  |0.0160|
|  - public_relations                   |      0|none  |     0|acc   |0.7636|±  |0.0407|
|  - security_studies                   |      0|none  |     0|acc   |0.8204|±  |0.0246|
|  - sociology                          |      0|none  |     0|acc   |0.8856|±  |0.0225|
|  - us_foreign_policy                  |      0|none  |     0|acc   |0.9100|±  |0.0288|
| - stem                                |N/A    |none  |     0|acc   |0.6505|±  |0.1266|
|  - abstract_algebra                   |      0|none  |     0|acc   |0.4100|±  |0.0494|
|  - anatomy                            |      0|none  |     0|acc   |0.6444|±  |0.0414|
|  - astronomy                          |      0|none  |     0|acc   |0.8224|±  |0.0311|
|  - college_biology                    |      0|none  |     0|acc   |0.8681|±  |0.0283|
|  - college_chemistry                  |      0|none  |     0|acc   |0.5500|±  |0.0500|
|  - college_computer_science           |      0|none  |     0|acc   |0.6200|±  |0.0488|
|  - college_mathematics                |      0|none  |     0|acc   |0.4200|±  |0.0496|
|  - college_physics                    |      0|none  |     0|acc   |0.5392|±  |0.0496|
|  - computer_security                  |      0|none  |     0|acc   |0.8300|±  |0.0378|
|  - conceptual_physics                 |      0|none  |     0|acc   |0.7362|±  |0.0288|
|  - electrical_engineering             |      0|none  |     0|acc   |0.7034|±  |0.0381|
|  - elementary_mathematics             |      0|none  |     0|acc   |0.5503|±  |0.0256|
|  - high_school_biology                |      0|none  |     0|acc   |0.8742|±  |0.0189|
|  - high_school_chemistry              |      0|none  |     0|acc   |0.6256|±  |0.0341|
|  - high_school_computer_science       |      0|none  |     0|acc   |0.8400|±  |0.0368|
|  - high_school_mathematics            |      0|none  |     0|acc   |0.4370|±  |0.0302|
|  - high_school_physics                |      0|none  |     0|acc   |0.5033|±  |0.0408|
|  - high_school_statistics             |      0|none  |     0|acc   |0.6944|±  |0.0314|
|  - machine_learning                   |      0|none  |     0|acc   |0.5982|±  |0.0465|

no i do not know why the stderr is high. plausibly it is due to the vllm backend used. this is my lm-eval command in most cases (works on h100):

lm_eval --model vllm --model_args pretrained=./miqu-1-70b-sf,tensor_parallel_size=4,dtype=auto,gpu_memory_utilization=0.88,data_parallel_size=2 --tasks mmlu --batch_size 20

The

tag is an essential component of HTML and is used to group and organize other HTML elements. It is a block-level element that can contain multiple elements such as paragraphs, headings, images, links, and other div elements. The use cases of the

tag are numerous, and it forms the backbone of structuring and styling web pages.

1. Organizing Content:
One of the primary use cases of the

tag is to organize and group content on a web page. It can be used to create sections for the header, main content, sidebar, and footer of a website. By using

tags to group related sections of the website, developers can apply styling and manipulate the layout of the content more effectively.

2. Styling and Layout:
The

tag is commonly used with CSS to control the layout and style of a web page. By assigning unique IDs or classes to

elements, developers can apply CSS rules to control the appearance of the content within the

. This includes setting background colors, borders, padding, margins, and other visual attributes.

3. Embedding Multimedia:
Developers often use the

tag to embed multimedia content such as videos, images, or audio files onto a web page. By creating a

container and placing the relevant source code within it, the multimedia content can be displayed in a specific location on the page.

4. Building Grid Systems:
The

tag is fundamental in modern web development for building grid systems. Developers use

elements to create responsive grid layouts that adjust based on the screen size and provide consistent content placement across various devices.

5. Scripting and Interactivity:
The

tag can serve as a container for JavaScript and other scripting languages. It enables the creation of interactive elements on the web page, from dropdown menus and tabs to carousels and accordions.

6. Integration with JavaScript Frameworks:
Modern JavaScript libraries and frameworks often make extensive use of

tags to render dynamic web applications. React, Angular, and Vue.js are examples of frameworks that use

tags as the building blocks for rendering and updating UI components.

7. Data Storage:
In some cases,

tags are used in conjunction with data attributes to store metadata about elements on the page. This data can then be accessed and manipulated by JavaScript.

8. Multi-Column Layouts:
Developers frequently use

tags to create multi-column layouts on web pages. By nesting

elements within one another, complex and responsive grid systems can be created to display content in multiple columns.

In conclusion, the

tag has a wide range of use cases in web development, including content organization, styling, layout control, multimedia embedding, scripting, grid systems, and more. It is a versatile and foundational element within HTML that enables developers to create dynamic and engaging web pages.

2024-02-01T01:47:43+01:00