Optional · AI analysis, AI questions, AI crash analysis · step by step
This page explains everything about the AI in the script: why it pays off, what speaks against it, how to create an account with a provider, what goes into the configuration, how to test, how to run the full analysis and open the report. Plus the features, the flow, which data leaves your computer and which models you can use. The AI works in three tools, each with its own model: analysis of reviews and figures, AI questions in the report and, new, the AI crash analysis for crashes from Apple and Android. Everything is set up in the browser-based setup.
Why AI?
The report counts stars, crashes and downloads. It tells you that an app is slipping. What really bothers users is in the text, and a language model reads that in seconds, for you, with evidence.
“Since the update the login hangs” appears in twelve reviews with three different wordings. The AI condenses that into one sentence, with quotes that appear verbatim in the reviews.
Three stars can mean “quite okay” or “I’m switching next week”. The AI distinguishes enthusiastic, satisfied, wish, annoyed and resigned, and counts how many threaten to leave.
At most five actions per app, with responsibility (development, support, provider, store, product), expected effect and the share of negative reviews they concern. The share is calculated by the program, not by the AI.
Pros and cons
The AI is deliberately optional. Without an enabled provider everything runs exactly as before, without a single request.
With "sendReviewText": false only stars, date, version, language and country go to the provider. Sentiment, quotes and reply drafts are then dropped, the rest stays.
Step 1 · Account with the provider
All five cloud providers work the same way: create an account, add a payment method or credit, generate an API key and copy it once. The key is shown only once. A normal chat subscription is not enough, it must be API access. The sixth card needs none of that: Ollama runs on your computer, without an account and without a key (setup below).
OPENAI_API_KEYANTHROPIC_API_KEYGEMINI_API_KEYPERPLEXITY_API_KEYXAI_API_KEYollama pull llama3.1:8bnoneRead before switching on: the provider’s API terms of use, especially whether inputs may be used for training. For the API access of the five providers this is usually excluded, unlike the free chat interfaces. The report’s privacy policy (section 4a) names recipients, fields and legal basis. The key never goes into a file of the script, only into the environment variable (next step).
Step 2 · Enter it in the configuration
config.json, one line in the shell.The ai block is ready in config/config.example.json. You copy it, set enabled to true for one provider and store the key as an environment variable. That is all. With local there is no key at all; the matching settings are in the section Local instead of cloud.
Insert the ai block from the example file into config/config.json and set "enabled": true for the provider you want. If several are enabled, the first in the list counts. You can keep the model (default) or change it.
"ai": {
"providers": [
{ "name": "anthropic",
"enabled": true,
"model": "claude-sonnet-5-5",
"apiKeyEnvironmentVariable":
"ANTHROPIC_API_KEY" },
{ "name": "openai", "enabled": false, … }
],
"language": "en",
"monthlyTokenLimit": 2000000,
"sendReviewText": true,
"features": { "summary": true, "sentiment": true,
"themes": true, "versions": true,
"comparison": true, "replyDrafts": true,
"measures": true, "questions": false }
}The script reads the variable’s name from apiKeyEnvironmentVariable. Put the key into your shell configuration, on the Mac usually ~/.zshrc, then open a new terminal. For the daily run via scheduler the variable goes into its environment (see BETRIEB.md).
# ~/.zshrc export ANTHROPIC_API_KEY="sk-ant-…" # check without showing the value test -n "$ANTHROPIC_API_KEY" && echo set
If the key accidentally ends up in config.json, the script rejects it. In logs, error messages and the report it is removed automatically.
All in the ai block, all optional:
language de or en: language of the AI texts model per provider, see models below monthlyTokenLimit hard brake per month sendReviewText false: numbers only, no texts features.* switch off single features questions free-text questions, off by default maxReviewsPerRequest 60 per batch requestTimeoutSeconds 180 servicePort 8765 for the question service
Free-text questions are off by default because every question costs a request. For the “AI questions” tab in the report set "questions": true.
Step 3 · Testing
Three commands, from harmless to real. The first two cost nothing and send nothing.
Shows the active provider with model, the month’s usage and the estimated cost. No network access. If it says “no provider active”, enabled or the environment variable is missing.
./ratings.sh ai-usage
Shows per app how many reviews would be sent and in how many batches. Nothing goes out. So you see before the first real run what it covers.
./ratings.sh analyze --dry-run
Debug mode creates invented apps and reviews in separate folders. The menu asks whether the AI should analyse the test data. That sends invented texts but costs real requests. Ideal for the first run without touching real data.
./debugmenu.sh
1 create test data
3 report in the browserStep 4 · The full analysis
As soon as a provider is active, the AI runs automatically with ./ratings.sh all and in the menu “Fetch everything”. Menu line and header show it. To start it on its own:
Reads the stored reviews, sends only new or changed ones, writes the results as ai-analysis.json per app into data/ and rebuilds the report. At the end there is the cost table per app.
./ratings.sh analyze
The AI suggests new topics and missing search words for known topics. The tool counts how many more reviews a word would match. Both go into config/themen.json only when you confirm them.
./ratings.sh ai-themes ./ratings.sh ai-themes --accept 2 ./ratings.sh ai-themes --accept-term 1 3
A question to the stored reviews, narrowed by app, period and stars. Needs "questions": true. Saved questions are listed by ai-questions.
./ratings.sh ask "Which apps mention the bonus programme?" --app bkk-herkulis ./ratings.sh ai-questions
./ratings.sh analyzeanalysis, rebuild report./ratings.sh analyze --dry-runshow, send nothing./ratings.sh ai-usageprovider, tokens, cost./ratings.sh ai-modelsmodels and prices./ratings.sh ai-themestopic suggestions./ratings.sh ask "…"free-text question./ratings.sh ai-questionssaved questions./ratings.sh ai-reportreport with question service./ratings.sh ai-servequestion service aloneStep 5 · Open the report
The report is a single HTML file and makes no network requests. The only exception is one you start yourself: a small question service on your computer that passes questions from the report to the provider.
Creates reports/index.html and opens it in the browser. All AI results are in it: summaries, actions, sentiment, wishes, saved answers. Only the input field for new questions is missing.
./ratings.sh report
Menu: main menu 3 · short menu 2Starts the service on 127.0.0.1, creates the report and opens it in one step. The “AI questions” tab gets an input field and the button “Choose a question …” with twenty prepared questions. Runs until you press Ctrl+C.
You change the standard questions yourself: they are in config/fragen.json, each question once in German (de) and English (en). Change, add or delete, done; the next report shows the new list. The file contains no secrets and may be versioned.
./ratings.sh ai-report
Menu: main menu 4 · short menu 3Only your computer, only while the terminal is open, only a report created during the run. A random token per start, 4 KB per request, one question at a time, the same monthly limit as always. The report on the company server has no service and shows saved answers only.
data/ai-service.json token, 0600,
deleted on exitWhich features
Eleven features, each with its place in the report and the switch in ai.features. Everything except the crash analysis follows the filter: customer, app, platform, period.
Three sentences: praise, criticism, what changed recently. Plus main praise, main criticism, assessment (product, update or provider problem) and verbatim quotes. features.summary
At most five actions per app with responsibility, expected effect and the share of negative reviews they concern. features.measures
Enthusiastic, satisfied, wish, annoyed, resigned, plus urgency. The sentiment strip above the table sets the filter. features.sentiment
Freely assigned keywords per review with count and negative share, plus suggestions for config/themen.json that are adopted only after confirmation. features.themes
The wishes from the reviews counted, with a trend against the 90 days before. features.sentiment
Which complaints are new since a release and which have disappeared; which occur only in the latest version. features.versions
Per customer or across all: share of annoyed users, main praise, main criticism, assessment as columns. features.comparison
Share of reviews in which users threaten to switch or delete, per app. features.sentiment
For open one- and two-star reviews a suggestion to copy. Nothing is ever sent. features.replyDrafts
Free-text questions to the filtered reviews, answer with confidence and quotes. Twenty prepared questions from config/fragen.json, freely editable there. The tab appears only while the question service runs. features.questions
Create requests from Apple or Android crash reports, submit one or several at once. Result: cause with confidence and verbatim evidence, location in the code, technical fix, steps to verify, as a ticket for Jira or mail. Known crashes are recognised and not sent again. More below and on its own page. features.crashReports
Every AI statement carries the “KI” mark and names provider, model and date. All numbers in the report, including shares and counts next to AI texts, are calculated by the program from the data. The AI never delivers a number.
New · AI crash analysis · Apple and Android
On the “AI crash reports” tab you create requests from crashes: platform, title and report are required; a note (iOS version, feature, architecture), the app and the project folder are optional. The AI is meant to find the bug, not to guess. That is why the tool computes all facts itself, and every statement of the AI must appear verbatim in the material. All the details, with privacy, settings and the prompts verbatim: AI crash analysis.
Paste the report from Xcode, App Store Connect or the Play Console. It is stored locally in data/crash-reports.json. Up to here nothing leaves the computer.
The parser reads exception type, signal, versions, the top frames and which of them belong to the app, detects missing symbolication and obfuscation, and builds the fingerprint: the crash’s error number.
If the error number is in data/crash-history.json, the request becomes a duplicate, takes over the old analysis and is never sent again.
“Analyse” first shows exactly the text that leaves, what was replaced, which source files go along and the estimated tokens. Only the confirmation sends.
Requests run one after another. Every AI quote is checked against report, note, code and reviews; invented ones are dropped, without evidence the confidence drops.
A second call checks the analysis against the same material. It may strike statements and lower the confidence, never raise it. Can be switched off, costs about double.
Error number, environment, cause with confidence, evidence, location, fix, steps to verify, open questions and what could not be checked. To the clipboard, printed or as PDF.
open, analysed, Analysed/PD (personal data had to be replaced, which should not happen in a crash report), error, duplicate.
With unsymbolicated Apple reports or obfuscated Android traces nobody can determine the cause seriously. The report says so beforehand instead of letting the AI guess.
Redaction recognises patterns. A name somebody types freely into the note is not recognised. Hence the preview: read, then send.
If you do not want code in the cloud, leave the folder list empty or use the local model for crash reports. Then everything stays on the computer.
Three tools, three models
Per provider you choose three models, in the setup in the browser (./ratings.sh admin, step AI). Nobody can change the model in the report; both tabs show which one applies. After saving, the setup reminds you to restart the question service.
Many reviews in batches, every day. A cheap, fast model is usually enough. providers[].model
Single questions to the filtered reviews. A stronger model pays off, the volume is small. Empty means: same as the analysis. providers[].questionsModel
Reading stack traces and code correctly is the hardest task. Choose the best available model here. Empty means: same as the analysis. providers[].crashModel
How it works
This is how an analysis runs, from the stored data to the “KI” mark in the report.
The daily fetch from Apple and Google runs as before. The AI only reads what is already stored.
What has been analysed stays in ai-analysis.json. An app’s summary is only rebuilt when new reviews arrive.
A character budget per request so no model context bursts, and a token limit per month that stops hard.
Drafts are never sent, topic suggestions wait for your confirmation, data is never deleted or overwritten.
Every run is in data/logs/collect.log, without keys. All prompts are in KI-PROMPTS.md verbatim.
Which data is used
This is what a review looks like when it leaves the computer. A running number instead of the store ID, no name, no metric.
sendReviewText: false)A question in the report refers to exactly the reviews the filter at the top currently shows. The report hands the selection to the question service (apps, period, stars, the IDs of the filtered reviews and the filter chips as description). The service uses it to select the reviews locally; sent are the latest up to maxReviewsPerQuestion (300) within the character budget. The IDs and the description stay on the computer. With several apps selected, the answer refers to their overall state. The answer states how many reviews were read, how reliable it is (reliable, indication, unclear) and backs it with quotes; a question without a verbatim source counts as “unclear”.
Models and prices · as of 2026-10-24
The AI sorts, summarises and quotes. The program checks every quote and every value. So what matters is not the strongest model but one that reliably delivers clean JSON and costs little. Prices in US dollars per million tokens, input and output, from ./ratings.sh ai-models.
Bars: input price of the default model, relative to the most expensive. For scale: the full analysis of 90 reviews costs a few cents with gpt-6-luna; with a top model at 10 / 50 $ about a hundred times that, without the summaries getting noticeably better.
| Provider | Model | Input | Output | Class | For this project |
|---|---|---|---|---|---|
| OpenAI | gpt-6-luna | 0.10 $ | 0.50 $ | cheap, default; reasoning model | Recommended to start. Cheapest model; the internal thinking tokens count as output but stay small for batches of 60 reviews. |
| OpenAI | gpt-6.1-sol | 2.00 $ | 10.00 $ | mid-range | Good, but twenty times the price of Luna. Only if Luna is noticeably off with your reviews. |
| OpenAI | gpt-6-astra | 10.00 $ | 50.00 $ | top model, expensive | Not needed. Oversized for sorting and summarising. |
| OpenAI | gpt-5.6-terra | 2.00 $ | 12.00 $ | previous mid-range | Possible, but no reason: Luna is newer and cheaper. |
| OpenAI | gpt-5.6-luna | 0.20 $ | 1.20 $ | previous, cheap | Fine as a fallback if gpt-6-luna is ever unavailable. |
| Anthropic | claude-sonnet-5-5 | 2.00 $ | 10.00 $ | mid-range, default | Good middle. Reliable with JSON and German texts, clean quotes. Noticeably more expensive than the cheap models. |
| Anthropic | claude-haiku-5-5 | 0.10 $ | 0.50 $ | cheap, fast | Recommended to start. Same price as Luna, up to 100k tokens per request, which is plenty with the script’s character budget. |
| Anthropic | claude-opus-5-5 | 4.00 $ | 20.00 $ | strong, pricier | Not needed. An advantage only with very long, convoluted reviews. |
| Anthropic | claude-fable-5-1 | 10.00 $ | 50.00 $ | top model, expensive | Not needed. A hundred times the price of Haiku for the same task. |
| Google Gemini | gemini-3.8-flash | 0.75 $ | 3.75 $ | default; introductory price until 2026-12-31 | Good middle. Fast, large context. Re-check the price after 2026-12-31. |
| Google Gemini | gemini-3.5-flash-lite | 0.30 $ | 2.50 $ | cheap, simpler | Fine for sentiment and topics; a bit thinner than Flash for summaries. |
| Google Gemini | gemini-3.1-pro | 2.00 $ | 12.00 $ | strong, pricier | Not needed for this task. |
| Perplexity | openai/gpt-5.6-terra | 2.00 $ | 12.00 $ | default; OpenAI model via Perplexity | Possible, no advantage: Perplexity passes the request on to OpenAI, its strength (web search) is not used. Only if you already have an account there. |
| Perplexity | anthropic/claude-sonnet-5-5 | 2.00 $ | 10.00 $ | Anthropic model via Perplexity | As above: the same model as directly at Anthropic, one more provider in between. |
| Perplexity | perplexity/sonar | 0.25 $ | 2.50 $ | own model, cheap | Cheap, but less proven for strict JSON. Check first with --dry-run and test data. |
| xAI | grok-4.7 | 2.00 $ | 6.00 $ | only public API model, default | Fine, mid-range price. No cheap variant, so less attractive to start with. |
| Local (Ollama etc.) | llama3.1:8b and others | 0 $ | 0 $ | runs on your computer | Free and without a recipient. Slower and weaker at summaries than the cloud models; model choice and setup in the section Local instead of cloud. |
The table above contains only the models for which the script knows a list price. Here are all models the five providers offer via their APIs according to current price overviews, in US dollars per million tokens (input / output). Every model of a provider works in the script; a “no” in the column “In the script” only means the list price is missing, which you enter yourself with priceInputPerMillion and priceOutputPerMillion. Sources are third-party price overviews, not the providers’ pages; check there before deciding.
| Provider | Model | Input | Output | In the script | Assessment |
|---|---|---|---|---|---|
| OpenAI | gpt-6-astra | 10.00 | 50.00 | yes | Top model. Not needed. |
| OpenAI | gpt-6.1-sol | 2.00 | 10.00 | yes | Mid-range. Good, but pricey for this task. |
| OpenAI | gpt-6-sol | 2.00 | 10.00 | no | Predecessor of 6.1 Sol, same price. No reason. |
| OpenAI | gpt-6-luna | 0.10 | 0.50 | yes | Cheapest model. Recommended to start. |
| OpenAI | gpt-5.6-sol | – | – | no | Previous generation; price not consistent across overviews. |
| OpenAI | gpt-5.6-terra | 2.00 | 12.00 | yes | Previous mid-range. Luna is newer and cheaper. |
| OpenAI | gpt-5.6-luna | 0.20 | 1.20 | yes | Fallback if 6 Luna is unavailable. |
| OpenAI | gpt-5.5-pro, gpt-5.4 | – | – | no | Older generation, partly only in overviews. Not needed. |
| Anthropic | claude-fable-5-1 | 10.00 | 50.00 | yes | Top model for long autonomous agents. Not needed. |
| Anthropic | claude-opus-5-5 | 4.00 | 20.00 | yes | Strong, pricier. An advantage only with very long reviews. |
| Anthropic | claude-sonnet-5-5 | 2.00 | 10.00 | yes | Good middle, reliable JSON. Script default. |
| Anthropic | claude-haiku-5-5 | 0.10 | 0.50 | yes | Up to 100k tokens per request; above that 0.50 / 2.50. Recommended to start. |
| Anthropic | claude-opus-5, claude-sonnet-5 | 5.00 / 2.00 | 25.00 / 10.00 | no | Predecessors, still available. No reason over 5.5. |
| Google Gemini | gemini-3.8-flash | 0.75 | 3.75 | yes | Introductory price until 2026-12-31, then double according to overviews. Good middle. |
| Google Gemini | gemini-3.7-flash, gemini-3.6-flash | 0.75 | 3.75 | no | Same price as 3.8 Flash; 3.8 is newer. |
| Google Gemini | gemini-3.5-flash | 1.50 | 9.00 | no | Price inconsistent across overviews. No reason over 3.8. |
| Google Gemini | gemini-3.5-flash-lite | 0.30 | 2.50 | yes | Fine for sentiment and topics. |
| Google Gemini | gemini-3-flash | 0.50 | 3.00 | no | Older, cheap. Possible. |
| Google Gemini | gemini-3.1-pro | 2.00 | 12.00 | yes | Up to 200k tokens; more above. Not needed. |
| Perplexity | openai/…, anthropic/…, google/…, xai/… | wie Anbieter | wie Anbieter | partly | Agent API passes to the model provider without markup. In the script: openai/gpt-5.6-terra, anthropic/claude-sonnet-5-5. |
| Perplexity | perplexity/sonar | 1.00 | 1.00 | yes (0.25 / 2.50) | Plus a fee per request (5 to 12 $ per 1,000). The script still lists the old price; enter it yourself. Web search is not used. |
| Perplexity | sonar-pro, sonar-reasoning-pro, sonar-deep-research | 3.00 / 2.00 / 2.00 | 15.00 / 8.00 / 8.00 | no | Meant for web search and research, no use here. Chat Completions interface sunset since 2026-09-27. |
| xAI | grok-4.7 | 2.00 | 6.00 | yes | Current model; up to 200k tokens, above 4 / 12. Fine. |
| xAI | grok-4.6, grok-4.5 | 2.00 | 6.00 | no | Predecessors, same price. No reason. |
| xAI | grok-4.3, grok-4.20 | 1.25 | 2.50 | no | Cheaper; possible, enter the price yourself. |
| xAI | grok-build-0.1 | 1.00 | 2.00 | no | Cheapest xAI model, meant for code. Check with test data first. |
| xAI | grok-4, grok-4-fast, grok-4.1-fast, grok-3 | – | – | no | Retired on 2026-05-15 according to overviews. |
In short: to start, gpt-6-luna or claude-haiku-5-5 (0.10 / 0.50 $ each). If you want more care in summaries, take claude-sonnet-5-5 or gemini-3.8-flash. Top models are not worth it here because the program checks the answer anyway and calculates the numbers itself. Other models of the same provider work too; then only the list price is missing, which you enter yourself with priceInputPerMillion and priceOutputPerMillion. The assessment is based on price lists and model class, not on a comparison test with your reviews. Prices without guarantee; the provider’s invoice is what counts.
Local instead of cloud · Ollama, LM Studio and others
With the provider local (default Ollama, other servers via url) everything stays in house: the reviews only go to 127.0.0.1, your own computer. It costs nothing but time and memory. What you should know: a laptop needs minutes instead of seconds per batch, and small models follow the strict JSON and the rule “verbatim quotes only” less reliably than the cloud models. The checks in the program catch that; the summaries then come out shorter.
Mac and Windows via the installer from ollama.com, Linux with one line. Ollama then runs in the background. You pull a model once, about 5 GB.
# Linux curl -fsSL https://ollama.com/install.sh | sh # pull a model (once), check ollama pull llama3.1:8b ollama list
Ollama starts models with a small window (4096 tokens). A batch of reviews does not fit; the beginning would be cut off silently. Set it once, restart Ollama.
# Mac launchctl setenv OLLAMA_CONTEXT_LENGTH 16384 # Linux: in the systemd unit Environment=OLLAMA_CONTEXT_LENGTH=16384 # Windows: system environment variable
No key, no environment variable. For local models smaller batches and more time per request; url only if the server runs on another port (LM Studio: 1234, llama.cpp: 8080); label is a free display name, for example “LM Studio”. The former name ollama still works.
"ai": {
"providers": [
{ "name": "local",
"enabled": true,
"model": "llama3.1:8b" }
],
"maxReviewsPerRequest": 15,
"maxCharsPerRequest": 24000,
"summaryReviews": 40,
"requestTimeoutSeconds": 600
}| Model | RAM | Class | For this project |
|---|---|---|---|
llama3.1:8b | from 8 GB | default | Recommended to start. Reliable JSON, decent German, runs on any current Mac or laptop. |
gemma3:12b | from 12 GB | better German | Good choice with 16 GB. More careful summaries, a bit slower. |
qwen3:14b | from 16 GB | strong, slow | Strongest summaries among the small models. Takes long; fine for the nightly daily run. |
mistral-small3.2 | from 16 GB | good middle | Balanced between speed and care. |
llama3.2:3b | from 4 GB | small, fast | Enough for sentiment and topics; shorter texts, quotes more often unusable. For old or weak machines. |
In short: Mac with Apple Silicon and 16 GB: gemma3:12b. Everything else: llama3.1:8b. Every model your local server offers (with Ollama every model fetched by ollama pull) works; its name goes into "model". The assessment is based on model class and experience with strict JSON, not on a comparison test with your reviews; the code has not yet been run against a live Ollama or LM Studio installation. Only addresses on your own computer (127.0.0.1, localhost) or https are allowed: a local server elsewhere in the company network works only encrypted, because the reviews would otherwise travel the network in the clear. The report’s privacy policy (section 4a) names no recipient for the local provider, because there is none.