Bump actions/checkout from 5 to 6

Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v6)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

.github/workflows/build-docker.yml

@@ -21,7 +21,7 @@ jobs:
             suffix: ""
             continue-on-error: false
     steps:
-      - uses: actions/checkout@v5
+      - uses: actions/checkout@v6
 
       - name: Get version number
         id: get_version
@@ -35,7 +35,7 @@ jobs:
           fi
 
       # Build Vue 3 frontend
-      - uses: actions/setup-node@v4
+      - uses: actions/setup-node@v6
         with:
           node-version: '22'
           cache: yarn

.github/workflows/ci.yml

@@ -12,15 +12,15 @@ jobs:
                 python-version: ["3.12"]
                 node-version: ["22"]
         steps:
-            - uses: actions/checkout@v5
-            - uses: awalsh128/cache-apt-pkgs-action@v1.5.3
+            - uses: actions/checkout@v6
+            - uses: awalsh128/cache-apt-pkgs-action@v1.6.0
               with:
                   packages: libsasl2-dev python3-dev libxml2-dev libxmlsec1-dev libxslt-dev libxmlsec1-openssl libxslt-dev libldap2-dev libssl-dev gcc musl-dev postgresql-dev zlib-dev jpeg-dev libwebp-dev openssl-dev libffi-dev cargo openldap-dev python3-dev xmlsec-dev xmlsec build-base g++ curl
                   version: 1.0
 
             # Setup python & dependencies
             - name: Set up Python ${{ matrix.python-version }}
-              uses: actions/setup-python@v5
+              uses: actions/setup-python@v6
               with:
                   python-version: ${{ matrix.python-version }}
                   cache: "pip"
@@ -43,7 +43,7 @@ jobs:
             # Build Vue frontend & Dependencies
             - name: Set up Node ${{ matrix.node-version }}
               if: steps.django_cache.outputs.cache-hit != 'true'
-              uses: actions/setup-node@v4
+              uses: actions/setup-node@v6
               with:
                   node-version: ${{ matrix.node-version }}
                   cache: "yarn"

.github/workflows/codeql-analysis.yml

@@ -12,7 +12,7 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v5
+        uses: actions/checkout@v6
         with:
           # We must fetch at least the immediate parents so that if this is
           # a pull request then we can checkout the head.
@@ -25,7 +25,7 @@ jobs:
 
       # Initializes the CodeQL tools for scanning.
       - name: Initialize CodeQL
-        uses: github/codeql-action/init@v3
+        uses: github/codeql-action/init@v4
         # Override language selection by uncommenting this and choosing your languages
         with:
           languages: python, javascript
@@ -47,6 +47,6 @@ jobs:
       #   make release
 
       - name: Perform CodeQL Analysis
-        uses: github/codeql-action/analyze@v3
+        uses: github/codeql-action/analyze@v4
         with:
           languages: javascript, python

.github/workflows/docs.yml

@@ -12,8 +12,8 @@ jobs:
     if: github.repository_owner == 'TandoorRecipes' && ${{ github.event.workflow_run.conclusion == 'success' }}
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v5
-      - uses: actions/setup-python@v5
+      - uses: actions/checkout@v6
+      - uses: actions/setup-python@v6
         with:
           python-version: 3.x
       - run: pip install mkdocs-material mkdocs-include-markdown-plugin

README.md

@@ -30,9 +30,11 @@
 ![Preview](docs/preview.png)
 
 ## Core Features
+
 - 🥗 **Manage your recipes** - Manage your ever growing recipe collection
 - 📆 **Plan** - multiple meals for each day
 - 🛒 **Shopping lists** - via the meal plan or straight from recipes
+- 🪄 **use AI** to recognize images, sort recipe steps, find nutrition facts and more
 - 📚 **Cookbooks** - collect recipes into books
 - 👪 **Share and collaborate** on recipes with friends and family
 
@@ -62,12 +64,13 @@ a public page.
 
 Documentation can be found [here](https://docs.tandoor.dev/).
 
-## Support our work
+## ❤️ Support our work ❤️
 Tandoor is developed by volunteers in their free time just because its fun. That said earning
 some money with the project allows us to spend more time on it and thus make improvements we otherwise couldn't.
 Because of that there are several ways you can support us
 
 - **GitHub Sponsors** You can sponsor contributors of this project on GitHub: [vabene1111](https://github.com/sponsors/vabene1111)
+- **Patron** You can sponsor contributors of this project on Patron: [vabene111](https://www.patreon.com/cw/vabene1111)
 - **Host at Hetzner** We have been very happy customers of Hetzner for multiple years for all of our projects. If you want to get into self-hosting or are tired of the expensive big providers, their cloud servers are a great place to get started. When you sign up via our [referral link](https://hetzner.cloud/?ref=ISdlrLmr9kGj) you will get 20€ worth of cloud credits and we get a small kickback too.
 - **Let us host for you** We are offering a [hosted version](https://app.tandoor.dev) where all profits support us and the development of tandoor (currently only available in germany).
 

boot.sh

@@ -2,7 +2,7 @@
 source venv/bin/activate
 
 # these are envsubst in the nginx config, make sure they default to something sensible when unset
-export TANDOOR_PORT="${TANDOOR_PORT:-8080}"
+export TANDOOR_PORT="${TANDOOR_PORT:-80}"
 export MEDIA_ROOT=${MEDIA_ROOT:-/opt/recipes/mediafiles};
 export STATIC_ROOT=${STATIC_ROOT:-/opt/recipes/staticfiles};
 
@@ -12,16 +12,18 @@ GUNICORN_LOG_LEVEL="${GUNICORN_LOG_LEVEL:-'info'}"
 
 PLUGINS_BUILD="${PLUGINS_BUILD:-0}"
 
-if [ "${TANDOOR_PORT}" -eq 80 ]; then
-    echo "TANDOOR_PORT set to 8080 because 80 is now taken by the integrated nginx"
-    TANDOOR_PORT=8080
-fi
-
 display_warning() {
     echo "[WARNING]"
     echo -e "$1"
 }
 
+# prepare nginx config
+envsubst '$MEDIA_ROOT $STATIC_ROOT $TANDOOR_PORT' < /opt/recipes/http.d/Recipes.conf.template > /opt/recipes/http.d/Recipes.conf
+
+# start nginx early to display error pages
+echo "Starting nginx"
+nginx
+
 echo "Checking configuration..."
 
 # SECRET_KEY (or a valid file at SECRET_KEY_FILE) must be set in .env file
@@ -93,7 +95,7 @@ fi
 
 echo "Collecting static files, this may take a while..."
 
-python manage.py collectstatic --noinput
+python manage.py collectstatic --noinput --clear
 
 echo "Done"
 
@@ -101,17 +103,6 @@ chmod -R 755 ${MEDIA_ROOT:-/opt/recipes/mediafiles}
 
 ipv6_disable=$(cat /sys/module/ipv6/parameters/disable)
 
-# prepare nginx config
-envsubst '$MEDIA_ROOT $STATIC_ROOT $TANDOOR_PORT' < /opt/recipes/http.d/Recipes.conf.template > /opt/recipes/http.d/Recipes.conf
-
-# start nginx
-echo "Starting nginx"
-nginx
-
 echo "Starting gunicorn"
-# Check if IPv6 is enabled, only then run gunicorn with ipv6 support
-if [ "$ipv6_disable" -eq 0 ]; then
-    exec gunicorn -b "[::]:$TANDOOR_PORT" --workers $GUNICORN_WORKERS --threads $GUNICORN_THREADS --access-logfile - --error-logfile - --log-level $GUNICORN_LOG_LEVEL recipes.wsgi
-else
-    exec gunicorn -b ":$TANDOOR_PORT" --workers $GUNICORN_WORKERS --threads $GUNICORN_THREADS --access-logfile - --error-logfile - --log-level $GUNICORN_LOG_LEVEL recipes.wsgi
-fi
+exec gunicorn --bind unix:/run/tandoor.sock --workers $GUNICORN_WORKERS --threads $GUNICORN_THREADS --timeout ${GUNICORN_TIMEOUT:-30} --access-logfile - --error-logfile - --log-level $GUNICORN_LOG_LEVEL recipes.wsgi
+

cookbook/admin.py

@@ -91,8 +91,8 @@ admin.site.register(SearchPreference, SearchPreferenceAdmin)
 
 
 class AiProviderAdmin(admin.ModelAdmin):
-    list_display = ('name', 'space', 'model',)
-    search_fields = ('name', 'space', 'model',)
+    list_display = ('name', 'space', 'model_name',)
+    search_fields = ('name', 'space', 'model_name',)
 
 
 admin.site.register(AiProvider, AiProviderAdmin)

cookbook/helper/ai_helper.py

@@ -33,12 +33,14 @@ class AiCallbackHandler(CustomLogger):
     space = None
     user = None
     ai_provider = None
+    function = None
 
-    def __init__(self, space, user, ai_provider):
+    def __init__(self, space, user, ai_provider, function):
         super().__init__()
         self.space = space
         self.user = user
         self.ai_provider = ai_provider
+        self.function = function
 
     def log_pre_api_call(self, model, messages, kwargs):
         pass
@@ -62,7 +64,7 @@ class AiCallbackHandler(CustomLogger):
             remaining_balance = self.space.ai_credits_balance - Decimal(str(credit_cost))
             if remaining_balance < 0:
                 remaining_balance = 0
-                if settings.HOSTED:
+                if settings.HOSTED and self.space.ai_credits_monthly == 0:
                     self.space.ai_enabled = False
 
             self.space.ai_credits_balance = remaining_balance
@@ -77,7 +79,7 @@ class AiCallbackHandler(CustomLogger):
             end_time=end_time,
             input_tokens=response_obj['usage']['prompt_tokens'],
             output_tokens=response_obj['usage']['completion_tokens'],
-            function=AiLog.F_FILE_IMPORT,
+            function=self.function,
             credit_cost=credit_cost,
             credits_from_balance=credits_from_balance,
         )

cookbook/helper/open_data_importer.py

@@ -51,10 +51,10 @@ class OpenDataImporter:
         for field in field_list:
             if isinstance(getattr(obj, field), float) or isinstance(getattr(obj, field), Decimal):
                 if abs(float(getattr(obj, field)) - float(existing_obj[field])) > 0.001:  # convert both to float and check if basically equal
-                    print(f'comparing FLOAT {obj} failed because field {field} is not equal ({getattr(obj, field)} != {existing_obj[field]})')
+                    #print(f'comparing FLOAT {obj} failed because field {field} is not equal ({getattr(obj, field)} != {existing_obj[field]})')
                     return False
             elif getattr(obj, field) != existing_obj[field]:
-                print(f'comparing {obj} failed because field {field} is not equal ({getattr(obj, field)} != {existing_obj[field]})')
+                #print(f'comparing {obj} failed because field {field} is not equal ({getattr(obj, field)} != {existing_obj[field]})')
                 return False
         return True
 
@@ -342,7 +342,7 @@ class OpenDataImporter:
                 'name': self.data[datatype][k]['name'],
                 'plural_name': self.data[datatype][k]['plural_name'] if self.data[datatype][k]['plural_name'] != '' else None,
                 'supermarket_category_id': self.slug_id_cache['category'][self.data[datatype][k]['store_category']] if self.data[datatype][k]['store_category'] in self.slug_id_cache['category'] else None,
-                'fdc_id': re.sub(r'\D', '', self.data[datatype][k]['fdc_id']) if self.data[datatype][k]['fdc_id'] != '' else None,
+                'fdc_id': re.sub(r'\D', '', str(self.data[datatype][k]['fdc_id'])) if self.data[datatype][k]['fdc_id'] != '' else None,
                 'open_data_slug': k,
                 'properties_food_unit_id': None,
                 'space_id': self.request.space.id,

cookbook/helper/property_helper.py

@@ -48,7 +48,7 @@ class FoodPropertyHelper:
                     found_property = False
                     # if food has a value for the given property type (no matter if conversion is possible)
                     has_property_value = False
-                    if i.food.properties_food_amount == 0 or i.food.properties_food_unit is None and not (i.amount == 0 or i.no_amount):  # if food is configured incorrectly
+                    if (i.food.properties_food_amount == 0 or i.food.properties_food_unit is None) and not (i.amount == 0 or i.no_amount):  # if food is configured incorrectly
                         computed_properties[pt.id]['food_values'][i.food.id] = {'id': i.food.id, 'food': {'id': i.food.id, 'name': i.food.name}, 'value': None}
                         computed_properties[pt.id]['missing_value'] = True
                     else:
@@ -56,15 +56,16 @@ class FoodPropertyHelper:
                             if p.property_type == pt and p.property_amount is not None:
                                 has_property_value = True
                                 for c in conversions:
-                                    if c.unit == i.food.properties_food_unit:
+                                    if c.unit == i.food.properties_food_unit and i.food.properties_food_amount != 0:
                                         found_property = True
                                         computed_properties[pt.id]['total_value'] += (c.amount / i.food.properties_food_amount) * p.property_amount
                                         computed_properties[pt.id]['food_values'] = self.add_or_create(
                                             computed_properties[p.property_type.id]['food_values'], c.food.id, (c.amount / i.food.properties_food_amount) * p.property_amount, c.food)
                     if not found_property:
                         # if no amount and food does not exist yet add it but don't count as missing
-                        if i.amount == 0 or i.no_amount and i.food.id not in computed_properties[pt.id]['food_values']:
-                            computed_properties[pt.id]['food_values'][i.food.id] = {'id': i.food.id, 'food': {'id': i.food.id, 'name': i.food.name}, 'value': 0}
+                        if i.amount == 0 or i.no_amount:
+                            if i.food.id not in computed_properties[pt.id]['food_values']:
+                                computed_properties[pt.id]['food_values'][i.food.id] = {'id': i.food.id, 'food': {'id': i.food.id, 'name': i.food.name}, 'value': 0}
                         # if amount is present but unit is missing indicate it in the result
                         elif i.unit is None:
                             if i.food.id not in computed_properties[pt.id]['food_values']:
@@ -72,7 +73,8 @@ class FoodPropertyHelper:
                             computed_properties[pt.id]['food_values'][i.food.id]['missing_unit'] = True
                         else:
                             computed_properties[pt.id]['missing_value'] = True
-                            computed_properties[pt.id]['food_values'][i.food.id] = {'id': i.food.id, 'food': {'id': i.food.id, 'name': i.food.name}, 'value': None}
+                            if i.food.id not in computed_properties[pt.id]['food_values']:
+                                computed_properties[pt.id]['food_values'][i.food.id] = {'id': i.food.id, 'food': {'id': i.food.id, 'name': i.food.name}, 'value': None}
                             if has_property_value and i.unit is not None:
                                 computed_properties[pt.id]['food_values'][i.food.id]['missing_conversion'] = {'base_unit': {'id': i.unit.id, 'name': i.unit.name}, 'converted_unit': {'id': i.food.properties_food_unit.id, 'name': i.food.properties_food_unit.name}}
 
@@ -82,8 +84,12 @@ class FoodPropertyHelper:
     # TODO move to central helper ? --> use defaultdict
     @staticmethod
     def add_or_create(d, key, value, food):
-        if key in d and d[key]['value']:
-            d[key]['value'] += value
+        if key in d:
+            # value can be None if a previous instance of the same food was missing a conversion
+            if d[key]['value']:
+                d[key]['value'] += value
+            else:
+                d[key]['value'] = value
         else:
             d[key] = {'id': food.id, 'food': {'id': food.id, 'name': food.name}, 'value': value}
         return d

cookbook/helper/recipe_search.py

@@ -288,7 +288,7 @@ class RecipeSearch():
 
     def _updated_on_filter(self):
         if self._updatedon:
-            self._queryset = self._queryset.filter(updated_at__date__date=self._updatedon)
+            self._queryset = self._queryset.filter(updated_at__date=self._updatedon)
         elif self._updatedon_lte:
             self._queryset = self._queryset.filter(updated_at__date__lte=self._updatedon_lte)
         elif self._updatedon_gte:
@@ -324,7 +324,7 @@ class RecipeSearch():
         self._queryset = self._queryset.annotate(recent=Coalesce(Max(Case(When(pk__in=num_recent_recipes.values('recipe'), then='viewlog__pk'))), Value(0)))
 
     def _favorite_recipes(self):
-        if self._sort_includes('favorite') or self._timescooked or self._timescooked_gte or self._timescooked_lte:
+        if self._sort_includes('favorite') or self._timescooked is not None or self._timescooked_gte is not None or self._timescooked_lte is not None:
             less_than = self._timescooked_lte and not self._sort_includes('-favorite')
             if less_than:
                 default = 1000
@@ -338,11 +338,11 @@ class RecipeSearch():
             )
             self._queryset = self._queryset.annotate(favorite=Coalesce(Subquery(favorite_recipes), default))
 
-        if self._timescooked:
-            self._queryset = self._queryset.filter(favorite=0)
-        elif self._timescooked_lte:
+        if self._timescooked is not None:
+            self._queryset = self._queryset.filter(favorite=self._timescooked)
+        elif self._timescooked_lte is not None:
             self._queryset = self._queryset.filter(favorite__lte=int(self._timescooked_lte)).exclude(favorite=0)
-        elif self._timescooked_gte:
+        elif self._timescooked_gte is not None:
             self._queryset = self._queryset.filter(favorite__gte=int(self._timescooked_gte))
 
     def keyword_filters(self, **kwargs):

cookbook/helper/recipe_url_import.py

@@ -69,15 +69,8 @@ def get_from_scraper(scrape, request):
     recipe_json['description'] = parse_description(description)
     recipe_json['description'] = automation_engine.apply_regex_replace_automation(recipe_json['description'], Automation.DESCRIPTION_REPLACE)
 
-    # assign servings attributes
-    try:
-        # dont use scrape.yields() as this will always return "x servings" or "x items", should be improved in scrapers directly
-        servings = scrape.schema.data.get('recipeYield') or 1
-    except Exception:
-        servings = 1
-
-    recipe_json['servings'] = parse_servings(servings)
-    recipe_json['servings_text'] = parse_servings_text(servings)
+    recipe_json['servings'] = parse_servings(scrape.schema.data.get('recipeYield'))
+    recipe_json['servings_text'] = parse_servings_text(scrape.schema.data.get('recipeYield'))
 
     # assign time attributes
     try:
@@ -155,7 +148,7 @@ def get_from_scraper(scrape, request):
 
     # assign steps
     try:
-        for i in parse_instructions(scrape.instructions()):
+        for i in parse_instructions(scrape.instructions_list()):
             recipe_json['steps'].append({
                 'instruction': i,
                 'ingredients': [],
@@ -177,11 +170,11 @@ def get_from_scraper(scrape, request):
         for x in scrape.ingredients():
             if x.strip() != '':
                 try:
-                    amount, unit, ingredient, note = ingredient_parser.parse(x)
+                    amount, unit, food, note = ingredient_parser.parse(x)
                     ingredient = {
                         'amount': amount,
                         'food': {
-                            'name': ingredient,
+                            'name': food,
                         },
                         'unit': None,
                         'note': note,
@@ -315,14 +308,29 @@ def clean_instruction_string(instruction):
     # handle unsupported, special UTF8 character in Thermomix-specific instructions,
     # that happen in nearly every recipe on Cookidoo, Zaubertopf Club, Rezeptwelt
     # and in Thermomix-specific recipes on many other sites
-    return normalized_string \
-        .replace("", _('reverse rotation')) \
-        .replace("", _('careful rotation')) \
-        .replace("", _('knead')) \
-        .replace("", _('thicken')) \
-        .replace("", _('warm up')) \
-        .replace("", _('ferment')) \
-        .replace("", _("sous-vide"))
+    normalized_string = normalized_string \
+        .replace(u"\uE003", _('reverse rotation')) \
+        .replace(u"\uE002", _('careful rotation')) \
+        .replace(u"\uE001", _('knead')) \
+        .replace(u"\uE031", _('thicken')) \
+        .replace(u"\uE019", _('warm up')) \
+        .replace(u"\uE02E", _('ferment')) \
+        .replace(u"\uE018", _('slow cook')) \
+        .replace(u"\uE033", _('egg boiler')) \
+        .replace(u"\uE016", _('kettle')) \
+        .replace(u"\uE01E", _('blend')) \
+        .replace(u"\uE011", _('pre-clean')) \
+        .replace(u"\uE026", _('high temperature')) \
+        .replace(u"\uE00D", _('rice cooker')) \
+        .replace(u"\uE00C", _('caramelize')) \
+        .replace(u"\uE038", _('peeler')) \
+        .replace(u"\uE037", _('slicer')) \
+        .replace(u"\uE036", _('grater')) \
+        .replace(u"\uE04C", _('spiralizer')) \
+        .replace(u"\uE02D", _("sous-vide"))
+
+
+    return normalized_string
 
 
 def parse_instructions(instructions):
@@ -391,7 +399,7 @@ def parse_servings(servings):
 def parse_servings_text(servings):
     if isinstance(servings, str):
         try:
-            servings = re.sub("\\d+", '', servings).strip()
+            servings = re.sub("\\d+", '', servings, 1).strip()
         except Exception:
             servings = ''
     if isinstance(servings, list):

cookbook/helper/shopping_helper.py

@@ -75,7 +75,8 @@ class RecipeShoppingEditor():
 
     @staticmethod
     def get_shopping_list_recipe(id, user, space):
-        return ShoppingListRecipe.objects.filter(id=id).filter(entries__space=space).filter(
+        # TODO this sucks since it wont find SLR's that no longer have any entries
+        return ShoppingListRecipe.objects.filter(id=id, space=space).filter(
             Q(entries__created_by=user)
             | Q(entries__created_by__in=list(user.get_shopping_share()))
         ).prefetch_related('entries').first()
@@ -136,7 +137,8 @@ class RecipeShoppingEditor():
             self.servings = servings
 
         self._delete_ingredients(ingredients=ingredients)
-        if self.servings != self._shopping_list_recipe.servings:
+        # need to check if there is a SLR because its possible it cant be found if all entries are deleted
+        if self._shopping_list_recipe and self.servings != self._shopping_list_recipe.servings:
             self.edit_servings()
         self._add_ingredients(ingredients=ingredients)
         return True

cookbook/helper/unit_conversion_helper.py

@@ -135,8 +135,9 @@ class UnitConversionHelper:
         :param food: base food
         :return: converted ingredient object from base amount/unit/food
         """
-        if uc.food is None or uc.food == food:
+        if (uc.food is None or uc.food == food) and uc.converted_amount > 0 and uc.base_amount > 0:
             if unit == uc.base_unit:
                 return Ingredient(amount=amount * (uc.converted_amount / uc.base_amount), unit=uc.converted_unit, food=food, space=self.space)
             else:
                 return Ingredient(amount=amount * (uc.base_amount / uc.converted_amount), unit=uc.base_unit, food=food, space=self.space)
+        return None

cookbook/integration/mealie1.py

@@ -96,14 +96,20 @@ class Mealie1(Integration):
                 self.import_log.msg += f"Ignoring {r['name']} because a recipe with this name already exists.\n"
                 self.import_log.save()
             else:
+                servings = 1
+                try:
+                    servings = r['recipe_servings'] if r['recipe_servings'] and r['recipe_servings'] != 0 else 1
+                except KeyError:
+                    pass
+
                 recipe = Recipe.objects.create(
                     waiting_time=parse_time(r['perform_time']),
                     working_time=parse_time(r['prep_time']),
                     description=r['description'][:512],
                     name=r['name'],
                     source_url=r['org_url'],
-                    servings=r['recipe_servings'] if r['recipe_servings'] and r['recipe_servings'] != 0 else 1,
-                    servings_text=r['recipe_yield'].strip() if r['recipe_yield'] else "",
+                    servings=servings,
+                    servings_text=r['recipe_yield'].strip()[:32] if r['recipe_yield'] else "",
                     internal=True,
                     created_at=r['created_at'],
                     space=self.request.space,
@@ -128,20 +134,32 @@ class Mealie1(Integration):
 
         steps_relation = []
         first_step_of_recipe_dict = {}
+        step_id_dict = {}
         for s in mealie_database['recipe_instructions']:
             if s['recipe_id'] in recipes_dict:
-                step = Step.objects.create(instruction=(s['text'] if s['text'] else "") + (f" \n {s['summary']}" if s['summary'] else ""),
+                step = Step.objects.create(instruction=(s['text'] if s['text'] else "") + (f" \n {s['summary']}" if 'summary' in s and s['summary'] else ""),
                                            order=s['position'],
                                            name=s['title'],
                                            space=self.request.space)
                 steps_relation.append(Recipe.steps.through(recipe_id=recipes_dict[s['recipe_id']], step_id=step.pk))
+                step_id_dict[s["id"]] = step.pk
                 if s['recipe_id'] not in first_step_of_recipe_dict:
                     first_step_of_recipe_dict[s['recipe_id']] = step.pk
 
+        # it is possible for a recipe to not have steps but have ingredients, in that case create an empty step to add them to later
+        for r in recipes_dict.keys():
+            if r not in first_step_of_recipe_dict:
+                step = Step.objects.create(instruction='',
+                                    order=0,
+                                    name='',
+                                    space=self.request.space)
+                steps_relation.append(Recipe.steps.through(recipe_id=recipes_dict[r], step_id=step.pk))
+                first_step_of_recipe_dict[r] = step.pk
+
         for n in mealie_database['notes']:
             if n['recipe_id'] in recipes_dict:
                 step = Step.objects.create(instruction=n['text'],
-                                           name=n['title'],
+                                           name=n['title'][:128] if n['title'] else "",
                                            order=100,
                                            space=self.request.space)
                 steps_relation.append(Recipe.steps.through(recipe_id=recipes_dict[n['recipe_id']], step_id=step.pk))
@@ -153,6 +171,11 @@ class Mealie1(Integration):
         self.import_log.msg += f"Importing {len(mealie_database["recipes_ingredients"])} ingredients...\n"
         self.import_log.save()
 
+        # mealie stores the reference to a step (instruction) from an ingredient (reference) in the recipe_ingredient_ref_link table
+        recipe_ingredient_ref_link_dict = {}
+        for ref in mealie_database['recipe_ingredient_ref_link']:
+            recipe_ingredient_ref_link_dict[ref["reference_id"]] = ref["instruction_id"]
+
         ingredients_relation = []
         for i in mealie_database['recipes_ingredients']:
             if i['recipe_id'] in recipes_dict:
@@ -162,18 +185,18 @@ class Mealie1(Integration):
                         is_header=True,
                         space=self.request.space,
                     )
-                    ingredients_relation.append(Step.ingredients.through(step_id=first_step_of_recipe_dict[i['recipe_id']], ingredient_id=title_ingredient.pk))
+                    ingredients_relation.append(Step.ingredients.through(step_id=get_step_id(i, first_step_of_recipe_dict, step_id_dict,recipe_ingredient_ref_link_dict), ingredient_id=title_ingredient.pk))
                 if i['food_id']:
                     ingredient = Ingredient.objects.create(
                         food_id=foods_dict[i['food_id']] if i['food_id'] in foods_dict else None,
                         unit_id=units_dict[i['unit_id']] if i['unit_id'] in units_dict else None,
                         original_text=i['original_text'],
                         order=i['position'],
-                        amount=i['quantity'],
+                        amount=i['quantity'] if i['quantity'] else 0,
                         note=i['note'],
                         space=self.request.space,
                     )
-                    ingredients_relation.append(Step.ingredients.through(step_id=first_step_of_recipe_dict[i['recipe_id']], ingredient_id=ingredient.pk))
+                    ingredients_relation.append(Step.ingredients.through(step_id=get_step_id(i, first_step_of_recipe_dict, step_id_dict,recipe_ingredient_ref_link_dict), ingredient_id=ingredient.pk))
                 elif i['note'].strip():
                     amount, unit, food, note = ingredient_parser.parse(i['note'].strip())
                     f = ingredient_parser.get_food(food)
@@ -186,7 +209,7 @@ class Mealie1(Integration):
                         original_text=i['original_text'],
                         space=self.request.space,
                     )
-                    ingredients_relation.append(Step.ingredients.through(step_id=first_step_of_recipe_dict[i['recipe_id']], ingredient_id=ingredient.pk))
+                    ingredients_relation.append(Step.ingredients.through(step_id=get_step_id(i, first_step_of_recipe_dict, step_id_dict,recipe_ingredient_ref_link_dict), ingredient_id=ingredient.pk))
         Step.ingredients.through.objects.bulk_create(ingredients_relation)
 
         self.import_log.msg += f"Importing {len(mealie_database["recipes_to_categories"]) + len(mealie_database["recipes_to_tags"])} category and keyword relations...\n"
@@ -226,7 +249,7 @@ class Mealie1(Integration):
             for r in mealie_database['recipe_nutrition']:
                 if r['recipe_id'] in recipes_dict:
                     for key in property_types_dict:
-                        if r[key]:
+                        if key in r and r[key]:
                             properties_relation.append(
                                 Property(property_type_id=property_types_dict[key].pk,
                                          property_amount=Decimal(str(r[key])) / (
@@ -340,3 +363,10 @@ class Mealie1(Integration):
 
     def get_file_from_recipe(self, recipe):
         raise NotImplementedError('Method not implemented in storage integration')
+
+
+def get_step_id(i, first_step_of_recipe_dict, step_id_dict, recipe_ingredient_ref_link_dict):
+    try:
+        return step_id_dict[recipe_ingredient_ref_link_dict[i['reference_id']]]
+    except KeyError:
+        return first_step_of_recipe_dict[i['recipe_id']]

cookbook/integration/mealmaster.py

@@ -63,7 +63,15 @@ class MealMaster(Integration):
         current_recipe = ''
 
         for fl in file.readlines():
-            line = fl.decode("windows-1250")
+            line = ""
+            try:
+                line = fl.decode("UTF-8")
+            except UnicodeDecodeError:
+                try:
+                    line = fl.decode("windows-1250")
+                except Exception as e:
+                    line = "ERROR DECODING LINE"
+
             if (line.startswith('MMMMM') or line.startswith('-----')) and 'meal-master' in line.lower():
                 if current_recipe != '':
                     recipe_list.append(current_recipe)

cookbook/locale/nl/LC_MESSAGES/django.po

@@ -13,7 +13,7 @@ msgstr ""
 "Project-Id-Version: PACKAGE VERSION\n"
 "Report-Msgid-Bugs-To: \n"
 "POT-Creation-Date: 2024-08-01 15:04+0200\n"
-"PO-Revision-Date: 2025-07-31 19:14+0000\n"
+"PO-Revision-Date: 2025-09-23 19:45+0000\n"
 "Last-Translator: Justin Straver <justin.straver@gmail.com>\n"
 "Language-Team: Dutch <http://translate.tandoor.dev/projects/tandoor/"
 "recipes-backend/nl/>\n"
@@ -22,14 +22,14 @@ msgstr ""
 "Content-Type: text/plain; charset=UTF-8\n"
 "Content-Transfer-Encoding: 8bit\n"
 "Plural-Forms: nplurals=2; plural=n != 1;\n"
-"X-Generator: Weblate 5.8.4\n"
+"X-Generator: Weblate 5.13.1\n"
 
 #: .\cookbook\forms.py:45
 msgid ""
 "Both fields are optional. If none are given the username will be displayed "
 "instead"
 msgstr ""
-"Beide velden zijn optioneel. Indien niks is opgegeven wordt de "
+"Beide velden zijn optioneel. Indien niets is opgegeven wordt de "
 "gebruikersnaam weergegeven"
 
 #: .\cookbook\forms.py:62 .\cookbook\forms.py:246
@@ -771,7 +771,7 @@ msgstr ""
 #: .\cookbook\templates\account\email_confirm.html:22
 #: .\cookbook\templates\generic\delete_template.html:72
 msgid "Confirm"
-msgstr "Bevestigen"
+msgstr "Bevestig"
 
 #: .\cookbook\templates\account\email_confirm.html:29
 #, python-format
@@ -1052,7 +1052,7 @@ msgstr "Beheer"
 #: .\cookbook\templates\base.html:351
 #: .\cookbook\templates\space_overview.html:25
 msgid "Your Spaces"
-msgstr "Jouw Spaces"
+msgstr "Jouw Ruimtes"
 
 #: .\cookbook\templates\base.html:362
 #: .\cookbook\templates\space_overview.html:6
@@ -1996,12 +1996,12 @@ msgstr "Eigenaar"
 
 #: .\cookbook\templates\space_overview.html:57
 msgid "Leave Space"
-msgstr "Verlaat Space"
+msgstr "Verlaat Ruimte"
 
 #: .\cookbook\templates\space_overview.html:78
 #: .\cookbook\templates\space_overview.html:88
 msgid "Join Space"
-msgstr "Sluit aan bij ruimte"
+msgstr "Sluit aan bij Ruimte"
 
 #: .\cookbook\templates\space_overview.html:81
 msgid "Join an existing space."
@@ -2237,7 +2237,7 @@ msgstr "Er bestaat geen {self.basename} met id {target}"
 
 #: .\cookbook\views\api.py:250 .\cookbook\views\api.py:251
 msgid "Cannot merge with child object!"
-msgstr "Kan niet met kindobject samenvoegen!"
+msgstr "Kan niet met sub object samenvoegen!"
 
 #: .\cookbook\views\api.py:288 .\cookbook\views\api.py:289
 #, python-brace-format
@@ -2592,7 +2592,7 @@ msgstr "Uitnodigingslink"
 
 #: .\cookbook\views\delete.py:168
 msgid "Space Membership"
-msgstr "Space Lidmaatschap"
+msgstr "Ruimte Lidmaatschap"
 
 #: .\cookbook\views\edit.py:84
 msgid "You cannot edit this storage!"

cookbook/migrations/0001_squashed_0227_space_ai_default_provider_and_more.py

@@ -16,7 +16,7 @@ import uuid
 from django.conf import settings
 from django.db import migrations, models
 from cookbook.models import SearchFields
-
+from django.contrib.postgres.operations import TrigramExtension, UnaccentExtension
 
 def allSearchFields():
     return list(SearchFields.objects.values_list('id', flat=True))
@@ -141,6 +141,8 @@ class Migration(migrations.Migration):
     ]
 
     operations = [
+        TrigramExtension(),
+        UnaccentExtension(),
         migrations.RunPython(create_default_groups),
         migrations.CreateModel(
             name='AiProvider',

cookbook/migrations/0229_alter_ailog_options_alter_aiprovider_options_and_more.py

@@ -0,0 +1,26 @@
+# Generated by Django 5.2.6 on 2025-09-24 17:20
+
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('cookbook', '0228_space_space_setup_completed'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='ailog',
+            options={'ordering': ('-created_at',)},
+        ),
+        migrations.AlterModelOptions(
+            name='aiprovider',
+            options={'ordering': ('id',)},
+        ),
+        migrations.AlterField(
+            model_name='storage',
+            name='token',
+            field=models.CharField(blank=True, max_length=4098, null=True),
+        ),
+    ]

cookbook/migrations/0230_auto_20250925_2056.py

@@ -0,0 +1,15 @@
+# Generated by Django 5.2.6 on 2025-09-25 18:56
+
+from django.db import migrations
+from django.contrib.postgres.operations import TrigramExtension, UnaccentExtension
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('cookbook', '0229_alter_ailog_options_alter_aiprovider_options_and_more'),
+    ]
+
+    operations = [
+        TrigramExtension(),
+        UnaccentExtension(),
+    ]

cookbook/migrations/0231_alter_aiprovider_options_alter_automation_options_and_more.py

@@ -0,0 +1,141 @@
+# Generated by Django 5.2.6 on 2025-09-30 18:47
+
+from django.db import migrations
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('cookbook', '0230_auto_20250925_2056'),
+    ]
+
+    operations = [
+        migrations.AlterModelOptions(
+            name='aiprovider',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='automation',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='bookmarkletimport',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='comment',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='connectorconfig',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='cooklog',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='customfilter',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='exportlog',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='food',
+            options={'ordering': ('name',)},
+        ),
+        migrations.AlterModelOptions(
+            name='importlog',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='invitelink',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='keyword',
+            options={'ordering': ('name',)},
+        ),
+        migrations.AlterModelOptions(
+            name='mealplan',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='mealtype',
+            options={'ordering': ('name',)},
+        ),
+        migrations.AlterModelOptions(
+            name='recipe',
+            options={'ordering': ('name',)},
+        ),
+        migrations.AlterModelOptions(
+            name='recipebook',
+            options={'ordering': ('name',)},
+        ),
+        migrations.AlterModelOptions(
+            name='recipeimport',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='sharelink',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='shoppinglistentry',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='shoppinglistrecipe',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='space',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='storage',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='supermarket',
+            options={'ordering': ('name',)},
+        ),
+        migrations.AlterModelOptions(
+            name='supermarketcategory',
+            options={'ordering': ('name',)},
+        ),
+        migrations.AlterModelOptions(
+            name='sync',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='synclog',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='telegrambot',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='unit',
+            options={'ordering': ('name',)},
+        ),
+        migrations.AlterModelOptions(
+            name='unitconversion',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='userfile',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='userspace',
+            options={'ordering': ('pk',)},
+        ),
+        migrations.AlterModelOptions(
+            name='viewlog',
+            options={'ordering': ('pk',)},
+        ),
+    ]

cookbook/migrations/0232_shoppinglist.py

@@ -0,0 +1,29 @@
+# Generated by Django 5.2.7 on 2025-11-30 10:19
+
+import cookbook.models
+import django.db.models.deletion
+import django_prometheus.models
+from django.db import migrations, models
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ('cookbook', '0231_alter_aiprovider_options_alter_automation_options_and_more'),
+    ]
+
+    operations = [
+        migrations.CreateModel(
+            name='ShoppingList',
+            fields=[
+                ('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
+                ('name', models.CharField(blank=True, default='', max_length=32)),
+                ('description', models.TextField(blank=True)),
+                ('color', models.CharField(blank=True, max_length=7, null=True)),
+                ('created_at', models.DateTimeField(auto_now_add=True)),
+                ('updated_at', models.DateTimeField(auto_now=True)),
+                ('space', models.ForeignKey(on_delete=django.db.models.deletion.CASCADE, to='cookbook.space')),
+            ],
+            bases=(django_prometheus.models.ExportModelOperationsMixin('shopping_list'), models.Model, cookbook.models.PermissionModelMixin),
+        ),
+    ]

cookbook/models.py

@@ -402,6 +402,9 @@ class Space(ExportModelOperationsMixin('space'), models.Model):
     def __str__(self):
         return self.name
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class AiProvider(models.Model):
     name = models.CharField(max_length=128)
@@ -417,9 +420,18 @@ class AiProvider(models.Model):
     created_at = models.DateTimeField(auto_now_add=True)
     updated_at = models.DateTimeField(auto_now=True)
 
+    def __str__(self):
+        return self.name
+
+    class Meta:
+        ordering = ('pk',)
+
 
 class AiLog(models.Model, PermissionModelMixin):
     F_FILE_IMPORT = 'FILE_IMPORT'
+    F_STEP_SORT = 'STEP_SORT'
+    F_FOOD_PROPERTIES = 'FOOD_PROPERTIES'
+    F_RECIPE_PROPERTIES = 'RECIPE_PROPERTIES'
 
     ai_provider = models.ForeignKey(AiProvider, on_delete=models.SET_NULL, null=True)
     function = models.CharField(max_length=64)
@@ -437,6 +449,12 @@ class AiLog(models.Model, PermissionModelMixin):
     created_at = models.DateTimeField(auto_now_add=True)
     updated_at = models.DateTimeField(auto_now=True)
 
+    def __str__(self):
+        return f"{self.function} {self.ai_provider.name} {self.created_at}"
+
+    class Meta:
+        ordering = ('-created_at',)
+
 
 class ConnectorConfig(models.Model, PermissionModelMixin):
     HOMEASSISTANT = 'HomeAssistant'
@@ -462,6 +480,9 @@ class ConnectorConfig(models.Model, PermissionModelMixin):
     space = models.ForeignKey(Space, on_delete=models.CASCADE)
     objects = ScopedManager(space='space')
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class UserPreference(models.Model, PermissionModelMixin):
     # Themes
@@ -565,6 +586,9 @@ class UserSpace(models.Model, PermissionModelMixin):
     created_at = models.DateTimeField(auto_now_add=True)
     updated_at = models.DateTimeField(auto_now=True)
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class Storage(models.Model, PermissionModelMixin):
     DROPBOX = 'DB'
@@ -578,7 +602,7 @@ class Storage(models.Model, PermissionModelMixin):
     )
     username = models.CharField(max_length=128, blank=True, null=True)
     password = models.CharField(max_length=128, blank=True, null=True)
-    token = models.CharField(max_length=512, blank=True, null=True)
+    token = models.CharField(max_length=4098, blank=True, null=True)
     url = models.URLField(blank=True, null=True)
     path = models.CharField(blank=True, default='', max_length=256)
     created_by = models.ForeignKey(User, on_delete=models.PROTECT)
@@ -589,6 +613,9 @@ class Storage(models.Model, PermissionModelMixin):
     def __str__(self):
         return self.name
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class Sync(models.Model, PermissionModelMixin):
     storage = models.ForeignKey(Storage, on_delete=models.PROTECT)
@@ -604,6 +631,9 @@ class Sync(models.Model, PermissionModelMixin):
     def __str__(self):
         return self.path
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class SupermarketCategory(models.Model, PermissionModelMixin, MergeModelMixin):
     name = models.CharField(max_length=128, validators=[MinLengthValidator(1)])
@@ -629,6 +659,7 @@ class SupermarketCategory(models.Model, PermissionModelMixin, MergeModelMixin):
             models.UniqueConstraint(fields=['space', 'name'], name='smc_unique_name_per_space'),
             models.UniqueConstraint(fields=['space', 'open_data_slug'], name='supermarket_category_unique_open_data_slug_per_space')
         ]
+        ordering = ('name',)
 
 
 class Supermarket(models.Model, PermissionModelMixin):
@@ -648,6 +679,7 @@ class Supermarket(models.Model, PermissionModelMixin):
             models.UniqueConstraint(fields=['space', 'name'], name='sm_unique_name_per_space'),
             models.UniqueConstraint(fields=['space', 'open_data_slug'], name='supermarket_unique_open_data_slug_per_space')
         ]
+        ordering = ('name',)
 
 
 class SupermarketCategoryRelation(models.Model, PermissionModelMixin):
@@ -679,6 +711,9 @@ class SyncLog(models.Model, PermissionModelMixin):
     def __str__(self):
         return f"{self.created_at}:{self.sync} - {self.status}"
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class Keyword(ExportModelOperationsMixin('keyword'), TreeModel, PermissionModelMixin):
     if SORT_TREE_BY_NAME:
@@ -696,6 +731,7 @@ class Keyword(ExportModelOperationsMixin('keyword'), TreeModel, PermissionModelM
             models.UniqueConstraint(fields=['space', 'name'], name='kw_unique_name_per_space')
         ]
         indexes = (Index(fields=['id', 'name']),)
+        ordering = ('name',)
 
 
 class Unit(ExportModelOperationsMixin('unit'), models.Model, PermissionModelMixin, MergeModelMixin):
@@ -727,6 +763,7 @@ class Unit(ExportModelOperationsMixin('unit'), models.Model, PermissionModelMixi
             models.UniqueConstraint(fields=['space', 'name'], name='u_unique_name_per_space'),
             models.UniqueConstraint(fields=['space', 'open_data_slug'], name='unit_unique_open_data_slug_per_space')
         ]
+        ordering = ('name',)
 
 
 class Food(ExportModelOperationsMixin('food'), TreeModel, PermissionModelMixin):
@@ -793,14 +830,7 @@ class Food(ExportModelOperationsMixin('food'), TreeModel, PermissionModelMixin):
         self.delete()
         return target
 
-    def delete(self):
-        if self.ingredient_set.all().exclude(step=None).count() > 0:
-            raise ProtectedError(self.name + _(" is part of a recipe step and cannot be deleted"), self.ingredient_set.all().exclude(step=None))
-        else:
-            return super().delete()
-
     # MP_Tree move uses raw SQL to execute move, override behavior to force a save triggering post_save signal
-
     def move(self, *args, **kwargs):
         super().move(*args, **kwargs)
         # treebeard bypasses ORM, need to explicity save to trigger post save signals retrieve the object again to avoid writing previous state back to disk
@@ -867,6 +897,7 @@ class Food(ExportModelOperationsMixin('food'), TreeModel, PermissionModelMixin):
             Index(fields=['id']),
             Index(fields=['name']),
         )
+        ordering = ('name',)
 
 
 class UnitConversion(ExportModelOperationsMixin('unit_conversion'), models.Model, PermissionModelMixin):
@@ -893,6 +924,7 @@ class UnitConversion(ExportModelOperationsMixin('unit_conversion'), models.Model
             models.UniqueConstraint(fields=['space', 'base_unit', 'converted_unit', 'food'], name='f_unique_conversion_per_space'),
             models.UniqueConstraint(fields=['space', 'open_data_slug'], name='unit_conversion_unique_open_data_slug_per_space')
         ]
+        ordering = ('pk',)
 
 
 class Ingredient(ExportModelOperationsMixin('ingredient'), models.Model, PermissionModelMixin):
@@ -1097,13 +1129,14 @@ class Recipe(ExportModelOperationsMixin('recipe'), models.Model, PermissionModel
         sub_food_recipes = Q(id__in=Food.objects.filter(ingredient__step__recipe__in=related_recipes).exclude(recipe=None).values_list('recipe'))
         return Recipe.objects.filter(Q(id__in=related_recipes.values_list('id')) | sub_step_recipes | sub_food_recipes)
 
-    class Meta():
+    class Meta:
         indexes = (
             GinIndex(fields=["name_search_vector"]),
             GinIndex(fields=["desc_search_vector"]),
             Index(fields=['id']),
             Index(fields=['name']),
         )
+        ordering = ('name',)
 
 
 class Comment(ExportModelOperationsMixin('comment'), models.Model, PermissionModelMixin):
@@ -1125,6 +1158,9 @@ class Comment(ExportModelOperationsMixin('comment'), models.Model, PermissionMod
     def __str__(self):
         return self.text
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class RecipeImport(models.Model, PermissionModelMixin):
     name = models.CharField(max_length=128)
@@ -1152,6 +1188,9 @@ class RecipeImport(models.Model, PermissionModelMixin):
         self.delete()
         return recipe
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class RecipeBook(ExportModelOperationsMixin('book'), models.Model, PermissionModelMixin):
     name = models.CharField(max_length=128)
@@ -1169,6 +1208,7 @@ class RecipeBook(ExportModelOperationsMixin('book'), models.Model, PermissionMod
 
     class Meta():
         indexes = (Index(fields=['name']),)
+        ordering = ('name',)
 
 
 class RecipeBookEntry(ExportModelOperationsMixin('book_entry'), models.Model, PermissionModelMixin):
@@ -1214,6 +1254,7 @@ class MealType(models.Model, PermissionModelMixin):
         constraints = [
             models.UniqueConstraint(fields=['space', 'name', 'created_by'], name='mt_unique_name_per_space'),
         ]
+        ordering = ('name',)
 
 
 class MealPlan(ExportModelOperationsMixin('meal_plan'), models.Model, PermissionModelMixin):
@@ -1241,6 +1282,9 @@ class MealPlan(ExportModelOperationsMixin('meal_plan'), models.Model, Permission
     def __str__(self):
         return f'{self.get_label()} - {self.from_date} - {self.meal_type.name}'
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class ShoppingListRecipe(ExportModelOperationsMixin('shopping_list_recipe'), models.Model, PermissionModelMixin):
     name = models.CharField(max_length=32, blank=True, default='')
@@ -1256,6 +1300,21 @@ class ShoppingListRecipe(ExportModelOperationsMixin('shopping_list_recipe'), mod
     def __str__(self):
         return f'Shopping list recipe {self.id} - {self.recipe}'
 
+    class Meta:
+        ordering = ('pk',)
+
+
+class ShoppingList(ExportModelOperationsMixin('shopping_list'), models.Model, PermissionModelMixin):
+    name = models.CharField(max_length=32, blank=True, default='')
+    description = models.TextField(blank=True)
+    color = models.CharField(max_length=7, blank=True, null=True)
+
+    created_at = models.DateTimeField(auto_now_add=True)
+    updated_at = models.DateTimeField(auto_now=True)
+
+    space = models.ForeignKey(Space, on_delete=models.CASCADE)
+    objects = ScopedManager(space='space')
+
 
 class ShoppingListEntry(ExportModelOperationsMixin('shopping_list_entry'), models.Model, PermissionModelMixin):
     list_recipe = models.ForeignKey(ShoppingListRecipe, on_delete=models.CASCADE, null=True, blank=True, related_name='entries')
@@ -1287,6 +1346,9 @@ class ShoppingListEntry(ExportModelOperationsMixin('shopping_list_entry'), model
         except AttributeError:
             return None
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class ShareLink(ExportModelOperationsMixin('share_link'), models.Model, PermissionModelMixin):
     recipe = models.ForeignKey(Recipe, on_delete=models.CASCADE)
@@ -1302,6 +1364,9 @@ class ShareLink(ExportModelOperationsMixin('share_link'), models.Model, Permissi
     def __str__(self):
         return f'{self.recipe} - {self.uuid}'
 
+    class Meta:
+        ordering = ('pk',)
+
 
 def default_valid_until():
     return date.today() + timedelta(days=14)
@@ -1325,6 +1390,9 @@ class InviteLink(ExportModelOperationsMixin('invite_link'), models.Model, Permis
     def __str__(self):
         return f'{self.uuid}'
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class TelegramBot(models.Model, PermissionModelMixin):
     token = models.CharField(max_length=256)
@@ -1339,6 +1407,9 @@ class TelegramBot(models.Model, PermissionModelMixin):
     def __str__(self):
         return f"{self.name}"
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class CookLog(ExportModelOperationsMixin('cook_log'), models.Model, PermissionModelMixin):
     recipe = models.ForeignKey(Recipe, on_delete=models.CASCADE)
@@ -1356,7 +1427,7 @@ class CookLog(ExportModelOperationsMixin('cook_log'), models.Model, PermissionMo
     def __str__(self):
         return self.recipe.name
 
-    class Meta():
+    class Meta:
         indexes = (
             Index(fields=['id']),
             Index(fields=['recipe']),
@@ -1365,6 +1436,7 @@ class CookLog(ExportModelOperationsMixin('cook_log'), models.Model, PermissionMo
             Index(fields=['created_by']),
             Index(fields=['created_by', 'rating']),
         )
+        ordering = ('pk',)
 
 
 class ViewLog(ExportModelOperationsMixin('view_log'), models.Model, PermissionModelMixin):
@@ -1378,13 +1450,14 @@ class ViewLog(ExportModelOperationsMixin('view_log'), models.Model, PermissionMo
     def __str__(self):
         return self.recipe.name
 
-    class Meta():
+    class Meta:
         indexes = (
             Index(fields=['recipe']),
             Index(fields=['-created_at']),
             Index(fields=['created_by']),
             Index(fields=['recipe', '-created_at', 'created_by']),
         )
+        ordering = ('pk',)
 
 
 class ImportLog(models.Model, PermissionModelMixin):
@@ -1405,6 +1478,9 @@ class ImportLog(models.Model, PermissionModelMixin):
     def __str__(self):
         return f"{self.created_at}:{self.type}"
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class ExportLog(models.Model, PermissionModelMixin):
     type = models.CharField(max_length=32)
@@ -1425,6 +1501,9 @@ class ExportLog(models.Model, PermissionModelMixin):
     def __str__(self):
         return f"{self.created_at}:{self.type}"
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class BookmarkletImport(ExportModelOperationsMixin('bookmarklet_import'), models.Model, PermissionModelMixin):
     html = models.TextField()
@@ -1435,6 +1514,9 @@ class BookmarkletImport(ExportModelOperationsMixin('bookmarklet_import'), models
     objects = ScopedManager(space='space')
     space = models.ForeignKey(Space, on_delete=models.CASCADE)
 
+    class Meta:
+        ordering = ('pk',)
+
 
 # field names used to configure search behavior - all data populated during data migration
 # other option is to use a MultiSelectField from https://github.com/goinnn/django-multiselectfield
@@ -1502,6 +1584,9 @@ class UserFile(ExportModelOperationsMixin('user_files'), models.Model, Permissio
     def __str__(self):
         return f'{self.name} (#{self.id})'
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class Automation(ExportModelOperationsMixin('automations'), models.Model, PermissionModelMixin):
     FOOD_ALIAS = 'FOOD_ALIAS'
@@ -1548,6 +1633,9 @@ class Automation(ExportModelOperationsMixin('automations'), models.Model, Permis
     objects = ScopedManager(space='space')
     space = models.ForeignKey(Space, on_delete=models.CASCADE)
 
+    class Meta:
+        ordering = ('pk',)
+
 
 class CustomFilter(models.Model, PermissionModelMixin):
     RECIPE = 'RECIPE'
@@ -1578,3 +1666,4 @@ class CustomFilter(models.Model, PermissionModelMixin):
         constraints = [
             models.UniqueConstraint(fields=['space', 'name'], name='cf_unique_name_per_space')
         ]
+        ordering = ('pk',)

cookbook/serializer.py

@@ -37,7 +37,7 @@ from cookbook.models import (Automation, BookmarkletImport, Comment, CookLog, Cu
                              ShareLink, ShoppingListEntry, ShoppingListRecipe, Space,
                              Step, Storage, Supermarket, SupermarketCategory,
                              SupermarketCategoryRelation, Sync, SyncLog, Unit, UnitConversion,
-                             UserFile, UserPreference, UserSpace, ViewLog, ConnectorConfig, SearchPreference, SearchFields, AiLog, AiProvider)
+                             UserFile, UserPreference, UserSpace, ViewLog, ConnectorConfig, SearchPreference, SearchFields, AiLog, AiProvider, ShoppingList)
 from cookbook.templatetags.custom_tags import markdown
 from recipes.settings import AWS_ENABLED, MEDIA_URL, EMAIL_HOST
 
@@ -151,19 +151,22 @@ class CustomOnHandField(serializers.Field):
         return instance
 
     def to_representation(self, obj):
-        if not self.context["request"].user.is_authenticated:
+        try:
+            if not self.context["request"].user.is_authenticated:
+                return []
+            shared_users = []
+            if c := caches['default'].get(f'shopping_shared_users_{self.context["request"].space.id}_{self.context["request"].user.id}', None):
+                shared_users = c
+            else:
+                try:
+                    shared_users = [x.id for x in list(self.context['request'].user.get_shopping_share())] + [self.context['request'].user.id]
+                    caches['default'].set(f'shopping_shared_users_{self.context["request"].space.id}_{self.context["request"].user.id}', shared_users, timeout=5 * 60)
+                    # TODO ugly hack that improves API performance significantly, should be done properly
+                except AttributeError:  # Anonymous users (using share links) don't have shared users
+                    pass
+            return obj.onhand_users.filter(id__in=shared_users).exists()
+        except AttributeError:
             return []
-        shared_users = []
-        if c := caches['default'].get(f'shopping_shared_users_{self.context["request"].space.id}_{self.context["request"].user.id}', None):
-            shared_users = c
-        else:
-            try:
-                shared_users = [x.id for x in list(self.context['request'].user.get_shopping_share())] + [self.context['request'].user.id]
-                caches['default'].set(f'shopping_shared_users_{self.context["request"].space.id}_{self.context["request"].user.id}', shared_users, timeout=5 * 60)
-                # TODO ugly hack that improves API performance significantly, should be done properly
-            except AttributeError:  # Anonymous users (using share links) don't have shared users
-                pass
-        return obj.onhand_users.filter(id__in=shared_users).exists()
 
     def to_internal_value(self, data):
         return data
@@ -335,17 +338,26 @@ class AiProviderSerializer(serializers.ModelSerializer):
         return super().create(validated_data)
 
     def update(self, instance, validated_data):
-        validated_data = self.handle_global_space_logic(validated_data)
+        validated_data = self.handle_global_space_logic(validated_data, instance=instance)
         return super().update(instance, validated_data)
 
-    def handle_global_space_logic(self, validated_data):
+    def handle_global_space_logic(self, validated_data, instance=None):
         """
         allow superusers to create AI providers without a space but make sure everyone else only uses their own space
         """
-        if ('space' not in validated_data or not validated_data['space']) and self.context['request'].user.is_superuser:
-            validated_data['space'] = None
+        if self.context['request'].user.is_superuser:
+            if ('space' not in validated_data or not validated_data['space']):
+                validated_data['space'] = None
+            else:
+                validated_data['space'] = self.context['request'].space
         else:
-            validated_data['space'] = self.context['request'].space
+            if instance:
+                validated_data['space'] = instance.space
+            else:
+                validated_data['space'] = self.context['request'].space
+
+        if 'log_credit_cost' in validated_data and not self.context['request'].user.is_superuser:
+            del validated_data['log_credit_cost']
 
         return validated_data
 
@@ -834,28 +846,31 @@ class FoodSerializer(UniqueFieldsMixin, WritableNestedModelSerializer, ExtendedR
 
     @extend_schema_field(bool)
     def get_substitute_onhand(self, obj):
-        if not self.context["request"].user.is_authenticated:
+        try:
+            if not self.context["request"].user.is_authenticated:
+                return []
+            shared_users = []
+            if c := caches['default'].get(
+                    f'shopping_shared_users_{self.context["request"].space.id}_{self.context["request"].user.id}', None):
+                shared_users = c
+            else:
+                try:
+                    shared_users = [x.id for x in list(self.context['request'].user.get_shopping_share())] + [
+                        self.context['request'].user.id]
+                    caches['default'].set(
+                        f'shopping_shared_users_{self.context["request"].space.id}_{self.context["request"].user.id}',
+                        shared_users, timeout=5 * 60)
+                    # TODO ugly hack that improves API performance significantly, should be done properly
+                except AttributeError:  # Anonymous users (using share links) don't have shared users
+                    pass
+            filter = Q(id__in=obj.substitute.all())
+            if obj.substitute_siblings:
+                filter |= Q(path__startswith=obj.path[:Food.steplen * (obj.depth - 1)], depth=obj.depth)
+            if obj.substitute_children:
+                filter |= Q(path__startswith=obj.path, depth__gt=obj.depth)
+            return Food.objects.filter(filter).filter(onhand_users__id__in=shared_users).exists()
+        except AttributeError:
             return []
-        shared_users = []
-        if c := caches['default'].get(
-                f'shopping_shared_users_{self.context["request"].space.id}_{self.context["request"].user.id}', None):
-            shared_users = c
-        else:
-            try:
-                shared_users = [x.id for x in list(self.context['request'].user.get_shopping_share())] + [
-                    self.context['request'].user.id]
-                caches['default'].set(
-                    f'shopping_shared_users_{self.context["request"].space.id}_{self.context["request"].user.id}',
-                    shared_users, timeout=5 * 60)
-                # TODO ugly hack that improves API performance significantly, should be done properly
-            except AttributeError:  # Anonymous users (using share links) don't have shared users
-                pass
-        filter = Q(id__in=obj.substitute.all())
-        if obj.substitute_siblings:
-            filter |= Q(path__startswith=obj.path[:Food.steplen * (obj.depth - 1)], depth=obj.depth)
-        if obj.substitute_children:
-            filter |= Q(path__startswith=obj.path, depth__gt=obj.depth)
-        return Food.objects.filter(filter).filter(onhand_users__id__in=shared_users).exists()
 
     def create(self, validated_data):
         name = validated_data['name'].strip()
@@ -1382,6 +1397,20 @@ class ShoppingListRecipeSerializer(serializers.ModelSerializer):
         read_only_fields = ('id', 'created_by',)
 
 
+class ShoppingListSerializer(SpacedModelSerializer, WritableNestedModelSerializer):
+
+    def create(self, validated_data):
+        validated_data['name'] = validated_data['name'].strip()
+        space = validated_data.pop('space', self.context['request'].space)
+        obj, created = ShoppingList.objects.get_or_create(name__iexact=validated_data['name'], space=space, defaults=validated_data)
+        return obj
+
+    class Meta:
+        model = ShoppingList
+        fields = ('id', 'name', 'description', 'color', 'created_at', 'updated_at',)
+        read_only_fields = ('id', 'created_at', 'updated_at',)
+
+
 class ShoppingListEntrySerializer(WritableNestedModelSerializer):
     food = FoodSerializer(allow_null=True)
     unit = UnitSerializer(allow_null=True, required=False)
@@ -1709,6 +1738,12 @@ class FdcQuerySerializer(serializers.Serializer):
     foods = FdcQueryFoodsSerializer(many=True)
 
 
+class GenericModelReferenceSerializer(serializers.Serializer):
+    id = serializers.IntegerField()
+    model = serializers.CharField()
+    name = serializers.CharField()
+
+
 # Export/Import Serializers
 
 class KeywordExportSerializer(KeywordSerializer):

cookbook/tests/api/test_api_food.py

@@ -236,42 +236,6 @@ def test_delete(u1_s1, u1_s2, obj_1, obj_tree_1):
         assert Food.find_problems() == ([], [], [], [], [])
 
 
-def test_integrity(u1_s1, recipe_1_s1):
-    with scopes_disabled():
-        assert Food.objects.count() == 10
-        assert Ingredient.objects.count() == 10
-        f_1 = Food.objects.first()
-
-    # deleting food will fail because food is part of recipe
-    r = u1_s1.delete(
-        reverse(
-            DETAIL_URL,
-            args={f_1.id}
-        )
-    )
-    assert r.status_code == 403
-
-    with scopes_disabled():
-        i_1 = f_1.ingredient_set.first()
-        # remove Ingredient that references Food from recipe step
-        i_1.step_set.first().ingredients.remove(i_1)
-        assert Food.objects.count() == 10
-        assert Ingredient.objects.count() == 10
-
-    # deleting food will succeed because its not part of recipe and delete will cascade to Ingredient
-    r = u1_s1.delete(
-        reverse(
-            DETAIL_URL,
-            args={f_1.id}
-        )
-    )
-    assert r.status_code == 204
-
-    with scopes_disabled():
-        assert Food.objects.count() == 9
-        assert Ingredient.objects.count() == 9
-
-
 def test_move(u1_s1, obj_tree_1, obj_2, obj_3, space_1):
     with scope(space=space_1):
         # for some reason the 'path' attribute changes between the factory and the test when using both obj_tree and obj

cookbook/tests/api/test_api_meal_plan.py

@@ -99,19 +99,19 @@ def test_list_filter(obj_1, u1_s1):
 
     response = json.loads(
         u1_s1.get(
-            f'{reverse(LIST_URL)}?from_date={(timezone.now() + timedelta(days=2)).strftime("%Y-%m-%d")}'
+            f'{reverse(LIST_URL)}?from_date={(timezone.localtime(timezone.now()) + timedelta(days=1)).strftime("%Y-%m-%d")}'
         ).content)['results']
     assert len(response) == 0
 
     response = json.loads(
         u1_s1.get(
-            f'{reverse(LIST_URL)}?to_date={(timezone.now() - timedelta(days=2)).strftime("%Y-%m-%d")}'
+            f'{reverse(LIST_URL)}?to_date={(timezone.localtime(timezone.now()) - timedelta(days=1)).strftime("%Y-%m-%d")}'
         ).content)['results']
     assert len(response) == 0
 
     response = json.loads(
         u1_s1.get(
-            f'{reverse(LIST_URL)}?from_date={(timezone.now() - timedelta(days=2)).strftime("%Y-%m-%d")}&to_date={(timezone.now() + timedelta(days=2)).strftime("%Y-%m-%d")}'
+            f'{reverse(LIST_URL)}?from_date={(timezone.localtime(timezone.now()) - timedelta(days=1)).strftime("%Y-%m-%d")}&to_date={(timezone.localtime(timezone.now()) + timedelta(days=1)).strftime("%Y-%m-%d")}'
         ).content)['results']
     assert len(response) == 1
 
@@ -153,8 +153,8 @@ def test_add(arg, request, u1_s2, recipe_1_s1, meal_type):
             'id': meal_type.id,
             'name': meal_type.name
         },
-        'from_date': (timezone.now()).strftime("%Y-%m-%d"),
-        'to_date': (timezone.now()).strftime("%Y-%m-%d"),
+        'from_date': (timezone.localtime(timezone.now())).strftime("%Y-%m-%d"),
+        'to_date': (timezone.localtime(timezone.now())).strftime("%Y-%m-%d"),
         'servings': 1,
         'title': 'test',
         'shared': []
@@ -196,8 +196,8 @@ def test_add_with_shopping(u1_s1, meal_type):
             'id': meal_type.id,
             'name': meal_type.name
         },
-        'from_date': (timezone.now()).strftime("%Y-%m-%d"),
-        'to_date': (timezone.now()).strftime("%Y-%m-%d"),
+        'from_date': (timezone.localtime(timezone.now())).strftime("%Y-%m-%d"),
+        'to_date': (timezone.localtime(timezone.now())).strftime("%Y-%m-%d"),
         'servings': 1,
         'title': 'test',
         'shared': [],
@@ -212,13 +212,13 @@ def test_add_with_shopping(u1_s1, meal_type):
 
 @pytest.mark.parametrize("arg", [
     ['', 2],
-    [f'?from_date={timezone.now().strftime("%Y-%m-%d")}', 1],
+    [f'?from_date={timezone.localtime(timezone.now()).strftime("%Y-%m-%d")}', 1],
     [
-        f'?to_date={(timezone.now() - timedelta(days=1)).strftime("%Y-%m-%d")}',
+        f'?to_date={(timezone.localtime(timezone.now()) - timedelta(days=1)).strftime("%Y-%m-%d")}',
         1
     ],
     [
-        f'?from_date={(timezone.now() + timedelta(days=2)).strftime("%Y-%m-%d")}&to_date={(timezone.now() + timedelta(days=2)).strftime("%Y-%m-%d")}',
+        f'?from_date={(timezone.localtime(timezone.now()) + timedelta(days=1)).strftime("%Y-%m-%d")}&to_date={(timezone.localtime(timezone.now()) + timedelta(days=1)).strftime("%Y-%m-%d")}',
         0
     ],
 ])

cookbook/tests/other/test_unit_conversion.py

@@ -185,3 +185,32 @@ def test_unit_conversions(space_1, space_2, u1_s1):
         assert next(x for x in conversions if x.unit == unit_kg_space_2) is not None
         assert abs(next(x for x in conversions if x.unit == unit_kg_space_2).amount - Decimal(0.1)) < 0.0001
         print(conversions)
+
+def test_conversion_with_zero(space_1, space_2, u1_s1):
+    with scopes_disabled():
+        uch = UnitConversionHelper(space_1)
+
+        unit_gram = Unit.objects.create(name='gram', base_unit='g', space=space_1)
+        unit_fantasy = Unit.objects.create(name='Fantasy Unit', base_unit=None, space=space_1)
+
+        food_1 = Food.objects.create(name='Test Food 1', space=space_1)
+
+        ingredient_food_1_gram = Ingredient.objects.create(
+            food=food_1,
+            unit=unit_gram,
+            amount=100,
+            space=space_1,
+        )
+
+        print('\n----------- TEST BASE CUSTOM CONVERSION - TO CUSTOM CONVERSION ---------------')
+        UnitConversion.objects.create(
+            base_amount=0,
+            base_unit=unit_gram,
+            converted_amount=0,
+            converted_unit=unit_fantasy,
+            space=space_1,
+            created_by=auth.get_user(u1_s1),
+        )
+        conversions = uch.get_conversions(ingredient_food_1_gram)
+
+        assert len(conversions) == 1 # conversion always includes the ingredient, if count is 1 no other conversion was found

cookbook/urls.py

@@ -40,6 +40,7 @@ router.register(r'recipe-book-entry', api.RecipeBookEntryViewSet)
 router.register(r'unit-conversion', api.UnitConversionViewSet)
 router.register(r'property-type', api.PropertyTypeViewSet)  # NOTE: if regenerating the legacy API these need renamed to food-property
 router.register(r'property', api.PropertyViewSet)
+router.register(r'shopping-list', api.ShoppingListViewSet)
 router.register(r'shopping-list-entry', api.ShoppingListEntryViewSet)
 router.register(r'shopping-list-recipe', api.ShoppingListRecipeViewSet)
 router.register(r'space', api.SpaceViewSet)
@@ -104,6 +105,7 @@ urlpatterns = [
     path('api/sync_all/', api.sync_all, name='api_sync'),
     path('api/recipe-from-source/', api.RecipeUrlImportView.as_view(), name='api_recipe_from_source'),
     path('api/ai-import/', api.AiImportView.as_view(), name='api_ai_import'),
+    path('api/ai-step-sort/', api.AiStepSortView.as_view(), name='api_ai_step_sort'),
     path('api/import-open-data/', api.ImportOpenData.as_view(), name='api_import_open_data'),
     path('api/ingredient-from-string/', api.ingredient_from_string, name='api_ingredient_from_string'),
     path('api/fdc-search/', api.FdcSearchView.as_view(), name='api_fdc_search'),

cookbook/views/api.py

@@ -9,6 +9,7 @@ import threading
 import traceback
 import uuid
 from collections import OrderedDict
+from functools import wraps
 from json import JSONDecodeError
 from urllib.parse import unquote
 from zipfile import ZipFile
@@ -19,12 +20,15 @@ import redis
 import requests
 from PIL import UnidentifiedImageError
 from django.contrib import messages
+from django.contrib.admin.utils import get_deleted_objects, NestedObjects
 from django.contrib.auth.models import Group, User
 from django.contrib.postgres.search import TrigramSimilarity
 from django.core.cache import caches
 from django.core.exceptions import FieldError, ValidationError
 from django.core.files import File
-from django.db.models import Case, Count, Exists, OuterRef, ProtectedError, Q, Subquery, Value, When
+from django.db import DEFAULT_DB_ALIAS
+from django.db.models import Case, Count, Exists, OuterRef, ProtectedError, Q, Subquery, Value, When, QuerySet
+from django.db.models.deletion import Collector
 from django.db.models.fields.related import ForeignObjectRel
 from django.db.models.functions import Coalesce, Lower
 from django.db.models.signals import post_save
@@ -84,7 +88,7 @@ from cookbook.models import (Automation, BookmarkletImport, ConnectorConfig, Coo
                              RecipeBookEntry, ShareLink, ShoppingListEntry,
                              ShoppingListRecipe, Space, Step, Storage, Supermarket, SupermarketCategory,
                              SupermarketCategoryRelation, Sync, SyncLog, Unit, UnitConversion,
-                             UserFile, UserPreference, UserSpace, ViewLog, RecipeImport, SearchPreference, SearchFields, AiLog, AiProvider
+                             UserFile, UserPreference, UserSpace, ViewLog, RecipeImport, SearchPreference, SearchFields, AiLog, AiProvider, ShoppingList
                              )
 from cookbook.provider.dropbox import Dropbox
 from cookbook.provider.local import Local
@@ -110,7 +114,7 @@ from cookbook.serializer import (AccessTokenSerializer, AutomationSerializer, Au
                                  LocalizationSerializer, ServerSettingsSerializer, RecipeFromSourceResponseSerializer, ShoppingListEntryBulkCreateSerializer, FdcQuerySerializer,
                                  AiImportSerializer, ImportOpenDataSerializer, ImportOpenDataMetaDataSerializer, ImportOpenDataResponseSerializer, ExportRequestSerializer,
                                  RecipeImportSerializer, ConnectorConfigSerializer, SearchPreferenceSerializer, SearchFieldsSerializer, RecipeBatchUpdateSerializer,
-                                 AiProviderSerializer, AiLogSerializer, FoodBatchUpdateSerializer
+                                 AiProviderSerializer, AiLogSerializer, FoodBatchUpdateSerializer, GenericModelReferenceSerializer, ShoppingListSerializer
                                  )
 from cookbook.version_info import TANDOOR_VERSION
 from cookbook.views.import_export import get_integration
@@ -303,7 +307,8 @@ class FuzzyFilterMixin(viewsets.ModelViewSet, ExtendedRecipeMixin):
                 filter = Q(name__icontains=query)
                 if self.request.user.is_authenticated:
                     if any([self.model.__name__.lower() in x for x in
-                            self.request.user.searchpreference.unaccent.values_list('field', flat=True)]):
+                            self.request.user.searchpreference.unaccent.values_list('field', flat=True)]) and (
+                            settings.DATABASES['default']['ENGINE'] == 'django.db.backends.postgresql'):
                         filter |= Q(name__unaccent__icontains=query)
 
                 self.queryset = (
@@ -368,11 +373,16 @@ class MergeMixin(ViewSetMixin):
                 isTree = False
 
             try:
+                # TODO these checks could be improved to merge existing properties and conversion in a smart way. For now it will just loose them to prevent duplicates
                 if isinstance(source, Food):
                     source.properties.all().delete()
                     source.properties.clear()
                     UnitConversion.objects.filter(food=source).delete()
 
+                if isinstance(source, Unit):
+                    UnitConversion.objects.filter(base_unit=source).delete()
+                    UnitConversion.objects.filter(converted_unit=source).delete()
+
                 for link in [field for field in source._meta.get_fields() if issubclass(type(field), ForeignObjectRel)]:
                     linkManager = getattr(source, link.get_accessor_name())
                     related = linkManager.all()
@@ -511,6 +521,143 @@ class TreeMixin(MergeMixin, FuzzyFilterMixin):
             return Response(content, status=status.HTTP_400_BAD_REQUEST)
 
 
+def paginate(func):
+    """
+    pagination decorator for custom ViewSet actions
+    """
+
+    @wraps(func)
+    def inner(self, *args, **kwargs):
+        queryset = func(self, *args, **kwargs)
+        assert isinstance(queryset, (list, QuerySet))
+
+        page = self.paginate_queryset(queryset)
+        if page is not None:
+            serializer = self.get_serializer(page, many=True)
+            return self.get_paginated_response(serializer.data)
+
+        serializer = self.get_serializer(queryset, many=True)
+        return Response(serializer.data)
+
+    return inner
+
+
+class DeleteRelationMixing:
+    """
+    mixin to add custom API function for model delete dependency checking
+    """
+
+    @staticmethod
+    def collect(obj):
+        # collector.nested() nested seems to not include protecting but does include cascading
+        # collector.protected: objects that raise Protected or Restricted error when deleting unit
+        # collector.field_updates: fields that get updated when deleting the unit
+        # collector.model_objs: collects the objects that should be deleted together with the selected unit
+
+        collector = NestedObjects(using=DEFAULT_DB_ALIAS)
+        collector.collect([obj])
+        return collector
+
+    @extend_schema(responses=GenericModelReferenceSerializer(many=True), parameters=[
+        OpenApiParameter(name='cache', description='If results can be cached or not', type=bool, default=True),
+    ])
+    @decorators.action(detail=True, methods=['GET'], serializer_class=GenericModelReferenceSerializer)
+    @paginate
+    def protecting(self, request, pk):
+        """
+        get a paginated list of objects that are protecting the selected object form being deleted
+        """
+        obj = self.queryset.filter(pk=pk, space=request.space).first()
+        if obj:
+            CACHE_KEY = f'DELETE_COLLECTOR_{request.space.pk}_PROTECTING_{obj.__class__.__name__}_{obj.pk}'
+            cache = self.request.query_params.get('cache', "true") == "true"
+            if caches['default'].has_key(CACHE_KEY) and cache:
+                return caches['default'].get(CACHE_KEY)
+
+            collector = self.collect(obj)
+            protected_objects = []
+            for o in collector.protected:
+                protected_objects.append({
+                    'id': o.pk,
+                    'model': o.__class__.__name__,
+                    'name': str(o),
+                })
+
+            caches['default'].set(CACHE_KEY, protected_objects, 60)
+            return protected_objects
+        else:
+            return []
+
+    @extend_schema(responses=GenericModelReferenceSerializer(many=True), parameters=[
+        OpenApiParameter(name='cache', description='If results can be cached or not', type=bool, default=True),
+    ])
+    @decorators.action(detail=True, methods=['GET'], serializer_class=GenericModelReferenceSerializer)
+    @paginate
+    def cascading(self, request, pk):
+        """
+        get a paginated list of objects that will be cascaded (deleted) when deleting the selected object
+        """
+        obj = self.queryset.filter(pk=pk, space=request.space).first()
+        if obj:
+            CACHE_KEY = f'DELETE_COLLECTOR_{request.space.pk}_CASCADING_{obj.__class__.__name__}_{obj.pk}'
+            cache = self.request.query_params.get('cache', "true") == "true"
+            if caches['default'].has_key(CACHE_KEY) and cache:
+                return caches['default'].get(CACHE_KEY)
+
+            collector = self.collect(obj)
+
+            cascading_objects = []
+            for model, objs in collector.model_objs.items():
+                for o in objs:
+                    if o.pk != pk and o.__class__.__name__ != obj.__class__.__name__:
+                        cascading_objects.append({
+                            'id': o.pk,
+                            'model': o.__class__.__name__,
+                            'name': str(o),
+                        })
+            caches['default'].set(CACHE_KEY, cascading_objects, 60)
+            return cascading_objects
+        else:
+            return []
+
+    @extend_schema(responses=GenericModelReferenceSerializer(many=True), parameters=[
+        OpenApiParameter(name='cache', description='If results can be cached or not', type=bool, default=True),
+    ])
+    @decorators.action(detail=True, methods=['GET'], serializer_class=GenericModelReferenceSerializer)
+    @paginate
+    def nulling(self, request, pk):
+        """
+        get a paginated list of objects where the selected object will be removed whe its deleted
+        """
+        obj = self.queryset.filter(pk=pk, space=request.space).first()
+        if obj:
+            CACHE_KEY = f'DELETE_COLLECTOR_{request.space.pk}_NULLING_{obj.__class__.__name__}_{obj.pk}'
+            cache = self.request.query_params.get('cache', "true") == "true"
+            if caches['default'].has_key(CACHE_KEY) and cache:
+                return caches['default'].get(CACHE_KEY)
+
+            collector = self.collect(obj)
+
+            nulling_objects = []
+            # field_updates is a dict of relations that will be updated and querysets of items affected
+            for key, value in collector.field_updates.items():
+                # iterate over each queryset for relation
+                for qs in value:
+                    # itereate over each object in queryset of relation
+                    for o in qs:
+                        nulling_objects.append(
+                            {
+                                'id': o.pk,
+                                'model': o.__class__.__name__,
+                                'name': str(o),
+                            }
+                        )
+            caches['default'].set(CACHE_KEY, nulling_objects, 60)
+            return nulling_objects
+        else:
+            return []
+
+
 @extend_schema_view(list=extend_schema(parameters=[
     OpenApiParameter(name='filter_list', description='User IDs, repeat for multiple', type=str, many=True),
 ]))
@@ -633,7 +780,7 @@ class SearchPreferenceViewSet(LoggingMixin, viewsets.ModelViewSet):
             return self.queryset.filter(user=self.request.user)
 
 
-class AiProviderViewSet(LoggingMixin, viewsets.ModelViewSet):
+class AiProviderViewSet(LoggingMixin, viewsets.ModelViewSet, DeleteRelationMixing):
     queryset = AiProvider.objects
     serializer_class = AiProviderSerializer
     permission_classes = [CustomAiProviderPermission & CustomTokenHasReadWriteScope]
@@ -656,7 +803,7 @@ class AiLogViewSet(LoggingMixin, viewsets.ModelViewSet):
         return self.queryset.filter(space=self.request.space)
 
 
-class StorageViewSet(LoggingMixin, viewsets.ModelViewSet):
+class StorageViewSet(LoggingMixin, viewsets.ModelViewSet, DeleteRelationMixing):
     # TODO handle delete protect error and adjust test
     queryset = Storage.objects
     serializer_class = StorageSerializer
@@ -667,7 +814,7 @@ class StorageViewSet(LoggingMixin, viewsets.ModelViewSet):
         return self.queryset.filter(space=self.request.space)
 
 
-class SyncViewSet(LoggingMixin, viewsets.ModelViewSet):
+class SyncViewSet(LoggingMixin, viewsets.ModelViewSet, DeleteRelationMixing):
     queryset = Sync.objects
     serializer_class = SyncSerializer
     permission_classes = [CustomIsAdmin & CustomTokenHasReadWriteScope]
@@ -728,7 +875,7 @@ class RecipeImportViewSet(LoggingMixin, viewsets.ModelViewSet):
         return Response({'msg': 'ok'}, status=status.HTTP_200_OK)
 
 
-class ConnectorConfigViewSet(LoggingMixin, viewsets.ModelViewSet):
+class ConnectorConfigViewSet(LoggingMixin, viewsets.ModelViewSet, DeleteRelationMixing):
     queryset = ConnectorConfig.objects
     serializer_class = ConnectorConfigSerializer
     permission_classes = [CustomIsAdmin & CustomTokenHasReadWriteScope]
@@ -738,7 +885,7 @@ class ConnectorConfigViewSet(LoggingMixin, viewsets.ModelViewSet):
         return self.queryset.filter(space=self.request.space)
 
 
-class SupermarketViewSet(LoggingMixin, StandardFilterModelViewSet):
+class SupermarketViewSet(LoggingMixin, StandardFilterModelViewSet, DeleteRelationMixing):
     queryset = Supermarket.objects
     serializer_class = SupermarketSerializer
     permission_classes = [CustomIsUser & CustomTokenHasReadWriteScope]
@@ -750,7 +897,7 @@ class SupermarketViewSet(LoggingMixin, StandardFilterModelViewSet):
 
 
 # TODO does supermarket category have settings to support fuzzy filtering and/or merge?
-class SupermarketCategoryViewSet(LoggingMixin, FuzzyFilterMixin, MergeMixin):
+class SupermarketCategoryViewSet(LoggingMixin, FuzzyFilterMixin, MergeMixin, DeleteRelationMixing):
     queryset = SupermarketCategory.objects
     model = SupermarketCategory
     serializer_class = SupermarketCategorySerializer
@@ -773,7 +920,7 @@ class SupermarketCategoryRelationViewSet(LoggingMixin, StandardFilterModelViewSe
         return super().get_queryset()
 
 
-class KeywordViewSet(LoggingMixin, TreeMixin):
+class KeywordViewSet(LoggingMixin, TreeMixin, DeleteRelationMixing):
     queryset = Keyword.objects
     model = Keyword
     serializer_class = KeywordSerializer
@@ -781,7 +928,7 @@ class KeywordViewSet(LoggingMixin, TreeMixin):
     pagination_class = DefaultPagination
 
 
-class UnitViewSet(LoggingMixin, MergeMixin, FuzzyFilterMixin):
+class UnitViewSet(LoggingMixin, MergeMixin, FuzzyFilterMixin, DeleteRelationMixing):
     queryset = Unit.objects
     model = Unit
     serializer_class = UnitSerializer
@@ -801,7 +948,7 @@ class FoodInheritFieldViewSet(LoggingMixin, viewsets.ReadOnlyModelViewSet):
         return super().get_queryset()
 
 
-class FoodViewSet(LoggingMixin, TreeMixin):
+class FoodViewSet(LoggingMixin, TreeMixin, DeleteRelationMixing):
     queryset = Food.objects
     model = Food
     serializer_class = FoodSerializer
@@ -947,6 +1094,82 @@ class FoodViewSet(LoggingMixin, TreeMixin):
             return JsonResponse({'msg': 'there was an error parsing the FDC data, please check the server logs'},
                                 status=500, json_dumps_params={'indent': 4})
 
+    @extend_schema(
+        parameters=[
+            OpenApiParameter(name='provider', description='ID of the AI provider that should be used for this AI request', type=int),
+        ]
+    )
+    @decorators.action(detail=True, methods=['POST'], )
+    def aiproperties(self, request, pk):
+        serializer = RecipeSerializer(data=request.data, partial=True, context={'request': request})
+        if serializer.is_valid():
+
+            if not request.query_params.get('provider', None) or not re.match(r'^(\d)+$', request.query_params.get('provider', None)):
+                response = {
+                    'error': True,
+                    'msg': _('You must select an AI provider to perform your request.'),
+                }
+                return Response(response, status=status.HTTP_400_BAD_REQUEST)
+
+            if not can_perform_ai_request(request.space):
+                response = {
+                    'error': True,
+                    'msg': _("You don't have any credits remaining to use AI or AI features are not enabled for your space."),
+                }
+                return Response(response, status=status.HTTP_400_BAD_REQUEST)
+
+            ai_provider = AiProvider.objects.filter(pk=request.query_params.get('provider')).filter(Q(space=request.space) | Q(space__isnull=True)).first()
+
+            litellm.callbacks = [AiCallbackHandler(request.space, request.user, ai_provider, AiLog.F_FOOD_PROPERTIES)]
+
+            property_type_list = list(PropertyType.objects.filter(space=request.space).values('id', 'name', 'description', 'unit', 'category', 'fdc_id'))
+            messages = [
+                {
+                    "role": "user",
+                    "content": [
+                        {
+                            "type": "text",
+                            "text": "Given the following food and the following different types of properties please update the food so that the properties attribute contains a list with all property types in the following format [{property_amount: <the property value>, property_type: {id: <the ID of the property type>, name: <the name of the property type>}}]."
+                                    "The property values should be in the unit given in the property type and for the amount specified in the properties_food_amount attribute of the food, which is given in the properties_food_unit."
+                                    "property_amount is a decimal number. Please try to keep a precision of two decimal places if given in your source data."
+                                    "Do not make up any data. If there is no data available for the given property type that is ok, just return null as a property_amount for that property type. Do not change anything else!"
+                                    "Most property types are likely going to be nutritional values. Please do not make up any values, only return values you can find in the sources available to you."
+                                    "Only return values if you are sure they are meant for the food given. Under no circumstance are you allowed to change any other value of the given food or change the structure in any way or form."
+                        },
+                        {
+                            "type": "text",
+                            "text": json.dumps(request.data)
+                        },
+                        {
+                            "type": "text",
+                            "text": json.dumps(property_type_list)
+                        },
+                    ]
+                },
+            ]
+
+            try:
+                ai_request = {
+                    'api_key': ai_provider.api_key,
+                    'model': ai_provider.model_name,
+                    'response_format': {"type": "json_object"},
+                    'messages': messages,
+                }
+                if ai_provider.url:
+                    ai_request['api_base'] = ai_provider.url
+                ai_response = completion(**ai_request)
+
+                response_text = ai_response.choices[0].message.content
+
+                return Response(json.loads(response_text), status=status.HTTP_200_OK)
+            except BadRequestError as err:
+                pass
+        response = {
+            'error': True,
+            'msg': 'The AI could not process your request. \n\n' + err.message,
+        }
+        return Response(response, status=status.HTTP_400_BAD_REQUEST)
+
     def destroy(self, *args, **kwargs):
         try:
             return (super().destroy(self, *args, **kwargs))
@@ -1049,7 +1272,7 @@ class FoodViewSet(LoggingMixin, TreeMixin):
     OpenApiParameter(name='order_direction', description='Order ascending or descending', type=str,
                      enum=['asc', 'desc']),
 ]))
-class RecipeBookViewSet(LoggingMixin, StandardFilterModelViewSet):
+class RecipeBookViewSet(LoggingMixin, StandardFilterModelViewSet, DeleteRelationMixing):
     queryset = RecipeBook.objects
     serializer_class = RecipeBookSerializer
     permission_classes = [(CustomIsOwner | CustomIsShared) & CustomTokenHasReadWriteScope]
@@ -1203,7 +1426,7 @@ class AutoPlanViewSet(LoggingMixin, mixins.CreateModelMixin, viewsets.GenericVie
         return Response(serializer.errors, 400)
 
 
-class MealTypeViewSet(LoggingMixin, viewsets.ModelViewSet):
+class MealTypeViewSet(LoggingMixin, viewsets.ModelViewSet, DeleteRelationMixing):
     """
     returns list of meal types created by the
     requesting user ordered by the order field.
@@ -1353,7 +1576,7 @@ class RecipePagination(PageNumberPagination):
     OpenApiParameter(name='filter', description=_('ID of a custom filter. Returns all recipes matched by that filter.'), type=int),
     OpenApiParameter(name='makenow', description=_('Filter recipes that can be made with OnHand food. [''true''/''<b>false</b>'']'), type=bool),
 ]))
-class RecipeViewSet(LoggingMixin, viewsets.ModelViewSet):
+class RecipeViewSet(LoggingMixin, viewsets.ModelViewSet, DeleteRelationMixing):
     queryset = Recipe.objects
     serializer_class = RecipeSerializer
     # TODO split read and write permission for meal plan guest
@@ -1588,6 +1811,82 @@ class RecipeViewSet(LoggingMixin, viewsets.ModelViewSet):
 
         return Response(serializer.errors, 400)
 
+    @extend_schema(
+        parameters=[
+            OpenApiParameter(name='provider', description='ID of the AI provider that should be used for this AI request', type=int),
+        ]
+    )
+    @decorators.action(detail=True, methods=['POST'], )
+    def aiproperties(self, request, pk):
+        serializer = RecipeSerializer(data=request.data, partial=True, context={'request': request})
+        if serializer.is_valid():
+
+            if not request.query_params.get('provider', None) or not re.match(r'^(\d)+$', request.query_params.get('provider', None)):
+                response = {
+                    'error': True,
+                    'msg': _('You must select an AI provider to perform your request.'),
+                }
+                return Response(response, status=status.HTTP_400_BAD_REQUEST)
+
+            if not can_perform_ai_request(request.space):
+                response = {
+                    'error': True,
+                    'msg': _("You don't have any credits remaining to use AI or AI features are not enabled for your space."),
+                }
+                return Response(response, status=status.HTTP_400_BAD_REQUEST)
+
+            ai_provider = AiProvider.objects.filter(pk=request.query_params.get('provider')).filter(Q(space=request.space) | Q(space__isnull=True)).first()
+
+            litellm.callbacks = [AiCallbackHandler(request.space, request.user, ai_provider, AiLog.F_RECIPE_PROPERTIES)]
+
+            property_type_list = list(PropertyType.objects.filter(space=request.space).values('id', 'name', 'description', 'unit', 'category', 'fdc_id'))
+            messages = [
+                {
+                    "role": "user",
+                    "content": [
+                        {
+                            "type": "text",
+                            "text": "Given the following recipe and the following different types of properties please update the recipe so that the properties attribute contains a list with all property types in the following format [{property_amount: <the property value>, property_type: {id: <the ID of the property type>, name: <the name of the property type>}}]."
+                                    "The property values should be in the unit given in the property type and calculated based on the total quantity of the foods used for the recipe."
+                                    "property_amount is a decimal number. Please try to keep a precision of two decimal places if given in your source data."
+                                    "Do not make up any data. If there is no data available for the given property type that is ok, just return null as a property_amount for that property type. Do not change anything else!"
+                                    "Most property types are likely going to be nutritional values. Please do not make up any values, only return values you can find in the sources available to you."
+                                    "Under no circumstance are you allowed to change any other value of the given food or change the structure in any way or form."
+                        },
+                        {
+                            "type": "text",
+                            "text": json.dumps(request.data)
+                        },
+                        {
+                            "type": "text",
+                            "text": json.dumps(property_type_list)
+                        },
+                    ]
+                },
+            ]
+
+            try:
+                ai_request = {
+                    'api_key': ai_provider.api_key,
+                    'model': ai_provider.model_name,
+                    'response_format': {"type": "json_object"},
+                    'messages': messages,
+                }
+                if ai_provider.url:
+                    ai_request['api_base'] = ai_provider.url
+                ai_response = completion(**ai_request)
+
+                response_text = ai_response.choices[0].message.content
+
+                return Response(json.loads(response_text), status=status.HTTP_200_OK)
+            except BadRequestError as err:
+                pass
+        response = {
+            'error': True,
+            'msg': 'The AI could not process your request. \n\n' + err.message,
+        }
+        return Response(response, status=status.HTTP_400_BAD_REQUEST)
+
     @extend_schema(responses=RecipeSerializer(many=False))
     @decorators.action(detail=True, pagination_class=None, methods=['PATCH'], serializer_class=RecipeSerializer)
     def delete_external(self, request, pk):
@@ -1635,7 +1934,7 @@ class UnitConversionViewSet(LoggingMixin, viewsets.ModelViewSet):
         enum=[m[0] for m in PropertyType.CHOICES])
     ]
 ))
-class PropertyTypeViewSet(LoggingMixin, viewsets.ModelViewSet):
+class PropertyTypeViewSet(LoggingMixin, viewsets.ModelViewSet, DeleteRelationMixing):
     queryset = PropertyType.objects
     serializer_class = PropertyTypeSerializer
     permission_classes = [CustomIsUser & CustomTokenHasReadWriteScope]
@@ -1712,6 +2011,17 @@ class ShoppingListRecipeViewSet(LoggingMixin, viewsets.ModelViewSet):
             return Response(serializer.errors, 400)
 
 
+class ShoppingListViewSet(LoggingMixin, viewsets.ModelViewSet, DeleteRelationMixing):
+    queryset = ShoppingList.objects
+    serializer_class = ShoppingListSerializer
+    permission_classes = [CustomIsUser & CustomTokenHasReadWriteScope]
+    pagination_class = DefaultPagination
+
+    def get_queryset(self):
+        queryset = self.queryset.filter(space=self.request.space).all()
+        return queryset
+
+
 @extend_schema_view(list=extend_schema(parameters=[
     OpenApiParameter(name='updated_after',
                      description=_('Returns only elements updated after the given timestamp in ISO 8601 format.'),
@@ -1866,7 +2176,7 @@ class BookmarkletImportViewSet(LoggingMixin, viewsets.ModelViewSet):
         return self.queryset.filter(space=self.request.space).all()
 
 
-class UserFileViewSet(LoggingMixin, StandardFilterModelViewSet):
+class UserFileViewSet(LoggingMixin, StandardFilterModelViewSet, DeleteRelationMixing):
     queryset = UserFile.objects
     serializer_class = UserFileSerializer
     permission_classes = [CustomIsUser & CustomTokenHasReadWriteScope]
@@ -2155,7 +2465,7 @@ class AiImportView(APIView):
 
             ai_provider = AiProvider.objects.filter(pk=serializer.validated_data['ai_provider_id']).filter(Q(space=request.space) | Q(space__isnull=True)).first()
 
-            litellm.callbacks = [AiCallbackHandler(request.space, request.user, ai_provider)]
+            litellm.callbacks = [AiCallbackHandler(request.space, request.user, ai_provider, AiLog.F_FILE_IMPORT)]
 
             messages = []
             uploaded_file = serializer.validated_data['file']
@@ -2264,6 +2574,13 @@ class AiImportView(APIView):
                     'msg': "Error parsing AI results. Response Text:\n\n" + response_text
                 }
                 return Response(RecipeFromSourceResponseSerializer(context={'request': request}).to_representation(response), status=status.HTTP_400_BAD_REQUEST)
+            except Exception:
+                traceback.print_exc()
+                response = {
+                    'error': True,
+                    'msg': "Error processing AI results. Response Text:\n\n" + response_text + "\n\n" + traceback.format_exc()
+                }
+                return Response(RecipeFromSourceResponseSerializer(context={'request': request}).to_representation(response), status=status.HTTP_400_BAD_REQUEST)
         else:
             response = {
                 'error': True,
@@ -2272,6 +2589,80 @@ class AiImportView(APIView):
             return Response(RecipeFromSourceResponseSerializer(context={'request': request}).to_representation(response), status=status.HTTP_400_BAD_REQUEST)
 
 
+class AiStepSortView(APIView):
+    throttle_classes = [AiEndpointThrottle]
+    permission_classes = [CustomIsUser & CustomTokenHasReadWriteScope]
+
+    @extend_schema(request=RecipeSerializer(many=False), responses=RecipeSerializer(many=False),
+                   parameters=[
+                       OpenApiParameter(name='provider', description='ID of the AI provider that should be used for this AI request', type=int),
+                   ])
+    def post(self, request, *args, **kwargs):
+        """
+        given an image or PDF file convert its content to a structured recipe using AI and the scraping system
+        """
+        serializer = RecipeSerializer(data=request.data, partial=True, context={'request': request})
+        if serializer.is_valid():
+
+            if not request.query_params.get('provider', None) or not re.match(r'^(\d)+$', request.query_params.get('provider', None)):
+                response = {
+                    'error': True,
+                    'msg': _('You must select an AI provider to perform your request.'),
+                }
+                return Response(response, status=status.HTTP_400_BAD_REQUEST)
+
+            if not can_perform_ai_request(request.space):
+                response = {
+                    'error': True,
+                    'msg': _("You don't have any credits remaining to use AI or AI features are not enabled for your space."),
+                }
+                return Response(response, status=status.HTTP_400_BAD_REQUEST)
+
+            ai_provider = AiProvider.objects.filter(pk=request.query_params.get('provider')).filter(Q(space=request.space) | Q(space__isnull=True)).first()
+
+            litellm.callbacks = [AiCallbackHandler(request.space, request.user, ai_provider, AiLog.F_STEP_SORT)]
+
+            messages = [
+                {
+                    "role": "user",
+                    "content": [
+                        {
+                            "type": "text",
+                            "text": "You are given data for a recipe formatted as json. You cannot under any circumstance change the value of any of the fields. You are only allowed to split the instructions into multiple steps and to sort the ingredients to their appropriate step. Your goal is to properly structure the recipe by splitting large instructions into multiple coherent steps and putting the ingredients that belong to this step into the ingredients list. Generally an ingredient of a cooking recipe should occur in the first step where its needed. Please sort the ingredients to the appropriate steps without changing any of the actual field values. Return the recipe in the same format you were given as json. Do not change any field value like strings or numbers, or change the sorting, also do not change the language."
+
+                        },
+                        {
+                            "type": "text",
+                            "text": json.dumps(request.data)
+                        },
+
+                    ]
+                },
+            ]
+
+            try:
+                ai_request = {
+                    'api_key': ai_provider.api_key,
+                    'model': ai_provider.model_name,
+                    'response_format': {"type": "json_object"},
+                    'messages': messages,
+                }
+                if ai_provider.url:
+                    ai_request['api_base'] = ai_provider.url
+                ai_response = completion(**ai_request)
+
+                response_text = ai_response.choices[0].message.content
+                # TODO validate by loading/dumping using serializer ?
+
+                return Response(json.loads(response_text), status=status.HTTP_200_OK)
+            except BadRequestError as err:
+                response = {
+                    'error': True,
+                    'msg': 'The AI could not process your request. \n\n' + err.message,
+                }
+                return Response(response, status=status.HTTP_400_BAD_REQUEST)
+
+
 class AppImportView(APIView):
     parser_classes = [MultiPartParser]
     throttle_classes = [RecipeImportThrottle]
@@ -2677,11 +3068,20 @@ def meal_plans_to_ical(queryset, filename):
     for p in queryset:
         event = Event()
         event['uid'] = p.id
-        event.add('dtstart', p.from_date)
+
+        start_date_time = p.from_date
+        end_date_time = p.from_date
+
         if p.to_date:
-            event.add('dtend', p.to_date)
-        else:
-            event.add('dtend', p.from_date)
+            end_date_time = p.to_date
+
+        if p.meal_type.time:
+            start_date_time = datetime.datetime.combine(p.from_date, p.meal_type.time)
+            end_date_time = datetime.datetime.combine(p.to_date, p.meal_type.time) + datetime.timedelta(minutes=60)
+
+        event.add('dtstart', start_date_time)
+        event.add('dtend', end_date_time)
+
         event['summary'] = f'{p.meal_type.name}: {p.get_label()}'
         event['description'] = p.note
         cal.add_component(event)

cookbook/views/views.py

@@ -43,7 +43,10 @@ def index(request, path=None, resource=None):
                 return HttpResponseRedirect(reverse_lazy('view_setup'))
 
     if 'signup_token' in request.session:
-        return HttpResponseRedirect(reverse('view_invite', args=[request.session.pop('signup_token', '')]))
+        value = request.session['signup_token']
+        del request.session['signup_token']
+        request.session.modified = True
+        return HttpResponseRedirect(reverse('view_invite', args=[value]))
 
     if request.user.is_authenticated or re.search(r'/recipe/\d+/', request.path[:512]) and request.GET.get('share'):
         return render(request, 'frontend/tandoor.html', {})

docs/contribute/related.md

@@ -33,4 +33,4 @@ Convert pictures of recipes to a structure that can be imported to Tandoor with
 
 Maintained by [smilerz](https://github.com/smilerz/tandoor-menu-generator)
 
-Generate a mealplan tbased on complex criteria and optionally insert it into an SVG menu template.
+Generate a meal plan based on complex criteria and optionally insert it into an SVG menu template.

docs/faq.md

@@ -36,7 +36,7 @@ then make sure you have set [all required headers](install/docker.md#required-he
 If that doesn't fix it, you can also refer to the appropriate sub section in the [reverse proxy documentation](install/docker.md#reverse-proxy) and verify your general webserver configuration.
 
 ### Required Headers
-Navigate to `/system` and review the headers listed in the DEBUG section.  At a minimum, if you are using a reverse proxy the headers must match the below conditions.
+Navigate to `/system/` and review the headers listed in the DEBUG section.  At a minimum, if you are using a reverse proxy the headers must match the below conditions.
 
 | Header      | Requirement |
 | :---        |    :----   |

docs/index.md

@@ -15,14 +15,14 @@
 <a href="https://discord.gg/RhzBrfWgtp" target="_blank" rel="noopener noreferrer"><img src="https://badgen.net/badge/icon/discord?icon=discord&label" ></a>
 <a href="https://hub.docker.com/r/vabene1111/recipes" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/docker/pulls/vabene1111/recipes" ></a>
 <a href="https://github.com/vabene1111/recipes/releases/latest" rel="noopener noreferrer"><img src="https://img.shields.io/github/v/release/vabene1111/recipes" ></a>
-<a href="https://app.tandoor.dev/accounts/login/?demo" rel="noopener noreferrer"><img src="https://img.shields.io/badge/demo-available-success" ></a>
+<a href="https://app.tandoor.dev/e/demo-auto-login/" rel="noopener noreferrer"><img src="https://img.shields.io/badge/demo-available-success" ></a>
 </p>
 
 <p align="center">
 <a href="https://tandoor.dev" target="_blank" rel="noopener noreferrer">Website</a> •
 <a href="https://docs.tandoor.dev/install/docker/" target="_blank" rel="noopener noreferrer">Installation</a> •
 <a href="https://docs.tandoor.dev/" target="_blank" rel="noopener noreferrer">Docs</a> •
-<a href="https://app.tandoor.dev/accounts/login/?demo" target="_blank" rel="noopener noreferrer">Demo</a> •
+<a href="https://app.tandoor.dev/e/demo-auto-login/" target="_blank" rel="noopener noreferrer">Demo</a> •
 <a href="https://community.tandoor.dev" target="_blank" rel="noopener noreferrer">Community</a> •
 <a href="https://discord.gg/RhzBrfWgtp" target="_blank" rel="noopener noreferrer">Discord</a>
 </p>

docs/install/docker.md

@@ -69,8 +69,6 @@ wget https://raw.githubusercontent.com/vabene1111/recipes/develop/docs/install/d
 
 Most deployments will likely use a reverse proxy.
 
-If your reverse proxy is not listed below, please refer to chapter [Others](#others).
-
 #### **Traefik**
 
 If you use Traefik, this configuration is the one for you.
@@ -115,6 +113,17 @@ wget https://raw.githubusercontent.com/vabene1111/recipes/develop/docs/install/d
 {% include "./docker/nginx-proxy/docker-compose.yml" %}
 ~~~
 
+
+#### **Apache proxy**
+
+If you use Apache as a reverse proxy, this configuration is the one for you.
+
+~~~yaml
+{% include "./docker/apache-proxy/docker-compose.yml" %}
+~~~
+
+Keep in mind, that the port configured for the service `web_recipes` should be the same as in chapter [Required Headers: Apache](#apache).
+
 ## **DockSTARTer**
 
 The main goal of [DockSTARTer](https://dockstarter.com/) is to make it quick and easy to get up and running with Docker.
@@ -139,7 +148,8 @@ if you manually change it/bind the folder as a volume.
 
 Please be sure to supply all required headers in your nginx/Apache/Caddy/... configuration!
 
-nginx:
+#### **nginx**
+
 ```nginx
 location / {
     proxy_set_header Host $http_host; # try $host instead if this doesn't work
@@ -149,7 +159,8 @@ location / {
 }
 ```
 
-Apache:
+#### **Apache**
+
 ```apache
 RequestHeader set X-Forwarded-Proto "https"
 Header always set Access-Control-Allow-Origin "*"
@@ -212,6 +223,6 @@ location /static/ {
 Tandoor 1 includes gunicorn, a python WSGI server that handles python code well but is not meant to serve mediafiles. Thus, it has always been recommended to set up a nginx webserver 
 (not just a reverse proxy) in front of Tandoor to handle mediafiles. The gunicorn server by default is exposed on port 8080.
 
-Tandoor 2 now occasionally bundles nginx inside the container and exposes port 80 where mediafiles are handled by nginx and all the other requests are (mostly) passed to gunicorn.
+Tandoor 2 now bundles nginx inside the container and exposes port 80 where mediafiles are handled by nginx and all the other requests are (mostly) passed to gunicorn.
 
 A [GitHub Issue](https://github.com/TandoorRecipes/recipes/issues/3851) has been created to allow for discussions and FAQ's on this issue while this change is fresh. It will later be updated in the docs here if necessary. 

docs/install/docker/apache-proxy/docker-compose.yml

@@ -0,0 +1,24 @@
+services:
+  db_recipes:
+    restart: always
+    image: postgres:16-alpine
+    volumes:
+      - ./postgresql:/var/lib/postgresql/data
+    env_file:
+      - ./.env
+
+  web_recipes:
+    restart: always
+    image: vabene1111/recipes
+    ports:
+      - 127.0.0.1:8080:80 # replace port
+    env_file:
+      - ./.env
+    volumes:
+      - staticfiles:/opt/recipes/staticfiles
+      - ./mediafiles:/opt/recipes/mediafiles
+    depends_on:
+      - db_recipes
+
+volumes:
+  staticfiles:

docs/install/manual.md

@@ -3,7 +3,7 @@
 These instructions are inspired from a standard django/gunicorn/postgresql instructions ([for example](https://www.digitalocean.com/community/tutorials/how-to-set-up-django-with-postgres-nginx-and-gunicorn-on-ubuntu-16-04))
 
 !!! warning
-    Make sure to use at least Python 3.10 (although 3.12 is preferred) or higher, and ensure that `pip` is associated with Python 3. Depending on your system configuration, using `python` or `pip` might default to Python 2. Make sure your machine has at least 2048 MB of memory; otherwise, the `yarn build` process may fail with the error: `FATAL ERROR: Reached heap limit - Allocation failed: JavaScript heap out of memory`.
+    Make sure to use at least Python 3.12 or higher, and ensure that `pip` is associated with Python 3. Depending on your system configuration, using `python` or `pip` might default to Python 2. Make sure your machine has at least 2048 MB of memory; otherwise, the `yarn build` process may fail with the error: `FATAL ERROR: Reached heap limit - Allocation failed: JavaScript heap out of memory`.
 
 !!! warning
     These instructions are **not** regularly reviewed and might be outdated.
@@ -77,10 +77,10 @@ Using binaries from the virtual env:
 /var/www/recipes/bin/pip3 install -r requirements.txt
 ```
 
-You will also need to install front end requirements and build them. For this navigate to the `./vue` folder and run
+You will also need to install front end requirements and build them. For this navigate to the `./vue3` folder and run
 
 ```shell
-cd ./vue
+cd ./vue3
 yarn install
 yarn build
 ```
@@ -224,7 +224,7 @@ bin/python3 manage.py migrate
 bin/python3 manage.py collectstatic --no-input
 bin/python3 manage.py collectstatic_js_reverse
 # change to frontend directory
-cd vue
+cd vue3
 # install and build frontend
 yarn install
 yarn build