Data integrity at the initial input stage is rarely straightforward, especially when diverse business logic requirements burden developers. Over time, this can lead to chaos in managing data structures. Another challenge lies in the libraries and tools available for handling these needs. To make them comprehensible and cover more complex requirements, they often resort to macros, which come with their own set of advantages and drawbacks—factors that may influence developers' preferences.

A prime example of such an infrastructure is the well-known Elixir library, Ecto. It demonstrates how to create a pipeline that processes input data from the start, cleanses it, and delivers a reliable output. This cleaned data can then be confidently stored in a database or displayed on a website.

The GuardedStruct library follows a similar path. It offers features such as builders, validation, and sanitization, enabling you to clean your data at the infrastructure level based on your business logic. Once processed, the data can be utilized across your application. Notably, during its development, the focus was on making it accessible even to project managers with limited Elixir knowledge. This ensures they can quickly build an MVP for their desired functionality with minimal learning curve.

Useful Links for This Article:

• Documentation and LiveBook: https://github.com/mishka-group/guarded_struct/blob/master/guidance/guarded-struct.livemd
• GitHub Project: https://github.com/mishka-group/guarded_struct
• Hex.pm Package: https://hex.pm/packages/guarded_struct

Why Did We Create the GuardedStruct Library?

In my personal experience, macros often feel like a magical black box. They can make a library difficult to understand, even for its creator, especially if the library relies heavily on macros. The lack of transparency in what happens under the hood requires a significant amount of investigation and study. This complexity was a key motivator behind our decision to build the GuardedStruct library, and sharing the story of how it came to be may clarify its purpose and usefulness.

About three years ago, the Mishka team participated in a private academic event. At the time, platforms like Bluesky and Mastodon had not yet gained widespread popularity. Our goal was to utilize the ActivityPub protocol to build a decentralized system for managing small restaurants using Telegram and Cloudflare's free services. We envisioned a hybrid system with a JavaScript-based frontend and backend to maximize the use of Cloudflare's free-tier capabilities. Additionally, the initial website (not a main platform but one focused on connecting users by proximity) was built using Elixir.

Upon delving into the ActivityPub protocol, we realized that each payload—whether exchanged between federation servers or sent from client to user—came in multiple formats. Worse, these formats had interdependencies between various fields. We started implementing one of these payloads with simple if and else statements. It quickly became apparent that even for a single section, we were writing at least 200 lines of code. Breaking it into smaller functions didn’t reduce the complexity significantly, as the sheer volume made it hard to maintain control. Non-technical contributors or those unfamiliar with Elixir found it nearly impossible to provide input during the research and development phases.

And that was just the beginning. We hadn’t yet encountered the deeply nested data structures inherent to the ActivityPub protocol. Over time, we optimized the library for each payload type until, about a year ago, the library was fully integrated into another project. Recently, we decided to separate it into its own independent project, giving rise to GuardedStruct.

Before Diving into the Features, Let’s Examine a Simple Key in ActivityPub

To better understand the challenges and how GuardedStruct addresses them, let’s take a look at a straightforward example of an ActivityPub payload. Even in its simplest form, the data can have multiple structures, as seen below:

Example 1: Basic Payload:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "summary": "Sally added an object",
  "type": "Add",
  "actor": {
    "type": "Person",
    "name": "Sally"
  },
  "object": "http://example.org/abc"
}

Example 2: Expanded Payload:

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "summary": "Sally added a picture of her cat to her cat picture collection",
  "type": "Add",
  "actor": {
    "type": "Person",
    "name": "Sally"
  },
  "object": {
    "type": "Image",
    "name": "A picture of my cat",
    "url": "http://example.org/img/cat.png"
  },
  "origin": {
    "type": "Collection",
    "name": "Camera Roll"
  },
  "target": {
    "type": "Collection",
    "name": "My Cat Pictures"
  }
}

These two payloads showcase how the same key, such as object, can vary significantly in structure. The first example uses a simple URL string for the object field, while the second example includes a nested structure with multiple properties. Additionally, new fields like origin and target appear in the second example, further increasing the complexity.

This variability makes processing and validating data tricky, especially in dynamic systems where payloads constantly evolve. Handling such cases effectively is a core challenge that GuardedStruct is designed to solve.

As you can see, while the API events and actions remain consistent, the data payloads can vary significantly in structure. In some cases, a single action can have up to five completely different data variations. Regardless of whether you're using a database like MongoDB or PostgreSQL, you must validate the data before storage. This validation is crucial both for adhering to business logic and ensuring data security and consistency.

This is precisely where GuardedStruct comes to your aid. It simplifies the process of validating and structuring your data to meet your application's requirements.

For testing and exploring the library, we’ve also prepared a sample implementation of this protocol (albeit a limited one), which you can check out here .

How Does the GuardedStruct Library Work?

At its core, GuardedStruct focuses on three main functionalities: Validation, Sanitization, and Construction (or Builder). Here's a breakdown of how these features work together:

• Constructor (Builder):

  The Constructor handles your business logic and dependencies by creating a structured representation of your data. During this process, data is first sanitized and then validated according to your defined policies. However, the Builder itself includes some built-in validation and sanitization to ensure the logic of your data remains intact.

  For example, one of the common attacks highlighted by OWASP is Mass Assignment, where attackers exploit vulnerabilities by injecting unexpected fields into data. GuardedStruct's Builder is designed to mitigate this type of attack right from the start.

• Validation and Type Enforcement:

  By leveraging macros, GuardedStruct performs extensive checks both at runtime and compile-time. This ensures data types and conditions are validated efficiently, making it a great fit for developers who want to implement @behaviour and create tailored specifications like t for modules or functions.

• Macros for Structs:

  To address the dynamic typing of Elixir Structs—where any data can be stored for a given key—the Builder uses three specialized macros, all under the umbrella of the guardedstruct macro:

  • field: Defines a basic key for your struct.
  • sub_field: Creates nested fields for hierarchical data.

  • conditional_field: Enables conditional logic for fields based on specific rules or requirements.

    These macros can work with data structures like lists, maps, or even deeply nested structures, providing flexibility for complex applications.

• Utility Functions:

  Each module that utilizes these macros automatically gains a set of helper functions. These functions allow you to:

  • Retrieve metadata about defined keys.
  • Enforce clear boundaries on how data is accessed and modified.
  • Manage and debug the data flow for each key.

  This combination of features ensures that your data handling logic is both robust and easy to maintain. GuardedStruct empowers developers to define data policies while reducing the risk of common vulnerabilities.

Example Usage of GuardedStruct

Here’s how you can work with GuardedStruct using built-in functions and macros. The library provides helper functions that simplify handling and validating keys within your struct. For example:

• YourModuleStruct.keys(): Retrieves all keys in the struct.
• YourModuleStruct.keys(:profile): Retrieves keys scoped to a specific field or namespace.
• YourModuleStruct.__information__(): Returns metadata and configuration details about the struct.
• YourModuleStruct.enforce_keys(): Enforces required keys across the entire struct.
• YourModuleStruct.enforce_keys(:profile): Enforces required keys for a specific field or section.

These utilities are pre-configured and can be tested or modified using the LiveBook example provided in the documentation. You can access it here .

Creating a Simple Struct with a Builder

To get started, you only need to define a module and use the GuardedStruct macro like so:

defmodule NormalStruct do
  use GuardedStruct

  guardedstruct do
    field(:field_one, String.t())
    field(:field_two, integer(), enforce: true)
    field(:field_three, boolean(), enforce: true)
    field(:field_four, atom(), default: :hey)
  end
end

Example Usage

Once defined, you can create and initialize the struct anywhere in your application:

NormalStruct.builder(%{field_one: "Mishka", field_two: 1, field_three: true})

Good News for Elixir Developers

GuardedStruct adheres to Elixir’s native concepts for working with structs, such as enforce , opaque , and module , while providing additional features to enhance flexibility and safety. This allows you to use familiar patterns with powerful extensions, which we’ll explore further in the next sections.

How to Define Validation for Each Field in GuardedStruct

When starting the GuardedStruct project, we initially forked the typed_struct library. However, it quickly became clear that the library’s creator no longer intended to maintain or update it, and its limited functionality couldn’t meet our requirements. Inspired by more advanced libraries in other languages, especially Rust and Scala, we designed a flexible and customizable validation system.

The goal was to give developers full control over field validation without enforcing overly strict rules. Let’s explore how you can define custom validation for each field:

Validation Rules

You can define a validator for any field. The basic requirement is that the input should consist of two elements:

• The value of the field.
• The field name.

The output must follow this format:

• Success: {:ok, :field_name, value}
• Error: {:error, :field_name, "Error message"}

Example Code

defmodule NormalValidatorStruct do
  use GuardedStruct

  # Define a struct with fields and attach a custom validator to the `:name` field
  guardedstruct do
    field(:name, String.t(), validator: {NormalValidatorStruct, :validator})
    field(:title, String.t())  # No validator here for simplicity
  end

  # Custom validator for the `:name` field
  def validator(:name, value) do
    if is_binary(value), do: {:ok, :name, value}, else: {:error, :name, "Not right!"}
  end

  # A default validator for testing purposes (not called in this example)
  def validator(field_name, value) do
    {:ok, field_name, value}
  end
end

Key Features

• Custom Validators per Field

  • For the :name field, we attach a validator using the macro: field(:name, String.t(), validator: {NormalValidatorStruct, :validator})
  • This ensures that any value passed to the :name field is validated using the :validator function.

• Default Module Lookup

  If you don’t define a validator in the macro itself, GuardedStruct will search for a matching validation function in the module. For example:

  def validator(:name, value) do
    # Custom validation logic
  end

  This eliminates the need to repeatedly attach the same validator in the macro, keeping your code clean and organized.

• Modularity

  Validators can also be defined in separate modules, making it easier to manage and reuse logic across different structs.

• Prioritization

  The validator defined in the macro takes precedence over other options. If no validator is specified in the macro, the module-level validator will be used.

A Practical Workflow

• Flexible Validators Define specific rules for each field based on your business logic.
• Clean Macros Place validation logic in the module to keep your guardedstruct macros clean and minimal.
• Error Handling Use the standardized format for success {:ok, :field, value} and error {:error, :field, "message"} to ensure consistent behavior throughout your application.
• Seamless Integration The system prioritizes flexibility while maintaining a clean architecture, allowing you to define and reuse validation logic effectively.

With this approach, GuardedStruct ensures your data validation logic remains clean, modular, and powerful, inspired by the robust practices of languages like Rust.

Can You Modify the Final Output After All Validations?

Absolutely! GuardedStruct is designed with flexibility in mind. While macros inherently introduce some structure, this library ensures that users aren’t restricted to predefined features. For this reason, a main_validator can be added to the parent macro guardedstruct . This allows you to define a function that runs additional validation or transformations on the final output after all field-levelvalidations are applied.

If you don’t specify a main_validator in the macro, the library will look for it in the module. This approach keeps your code modular and adaptable.

Example: Applying a Main Validator

defmodule NormalMainValidatorStruct do
  use GuardedStruct

  # Attach a main_validator to the struct
  guardedstruct main_validator: {__MODULE__, :main_validator} do
    field(:name, String.t())
    field(:title, String.t())
  end

  # Define the main_validator function
  def main_validator(value) do
    if Map.get(value, :title) == "mishka" do
      {:ok, value}
    else
      {:error, [%{message: "Not right!", field: :title, action: :validator}]}
    end
  end
end

Key Features of Main Validator

• Runs on Final Output The main_validator applies to the entire struct after all individual field-level validations have been processed.

• Customizable Location

  • You can define the main_validator directly in the guardedstruct macro: guardedstruct main_validator: {ModuleName, :main_validator} do ... end
  • Alternatively, you can simply place the function in the module, and it will be detected automatically.

• Standardized Output

  • For valid results, the function should return: {:ok, value}

  • For invalid results, it should return a list of errors with the following keys:

    message: A descriptive error message.
    field: The field that caused the error.
    action: The type of validation or action that failed.
    Example: {:error, [%{message: "Not right!", field: :title, action: :validator}]}

  
  
• Flexible Application The main_validator can be defined within the same module or called from another module, allowing for reusable and modular validation logic.

The practical workflow for validation involves several key steps. First, define field-level validators by using individual validator functions to handle specific rules for each field. Then, apply the main_validator for cross-field validation or any transformations required on the entire struct. Finally, ensure that the main_validator returns clear and structured results, either as success {:ok, value} or error responses {:error, [...]} , to facilitate seamless integration into your application.

Defining Validation and Sanitizers with the Derive Concept in GuardedStruct

The derive concept in GuardedStruct, inspired by Rust, allows you to apply both sanitization and validation in a streamlined and declarative manner. This approach ensures that data passes through sanitization first before undergoing validation, highlighting errors for any unmet conditions.

Example: Using Derive for Sanitization and Validation

defmodule NormalDeriveStruct do
  use GuardedStruct

  guardedstruct do
    field(:id, integer(), derive: "sanitize(trim) validate(integer, max_len=20, min_len=5)")
    field(:title, String.t(), derive: "sanitize(trim, upcase) validate(not_empty_string)")
    field(:name, String.t(),
      derive: "sanitize(trim, capitalize) validate(string, not_empty, max_len=20)"
    )
  end
end

This approach offers several benefits. It enhances control by allowing developers to introduce complex validation logic without needing to modify individual fields. It also promotes modularity by keeping validation logic in separate functions or modules, ensuring that your code remains clean and maintainable. Additionally, the standardized output format provides consistency, allowing errors to be handled uniformly throughout the application.

By using the main_validator, GuardedStruct empowers you to enforce comprehensive and customizable validation for your data, all while maintaining the flexibility and simplicity that Elixir developers love.

Breakdown of the Derive Workflow

The Derive Workflow involves a step-by-step process for sanitizing and validating fields. For the :id field, the system sanitizes the input by trimming leading and trailing whitespace. Validation ensures that the value is an integer and falls within a range of 5 to 20.

The :title field undergoes sanitization by removing spaces with trim and converting the string to uppercase using upcase. Validation checks that the field is a non-empty string. Similarly, the :name field is sanitized by trimming whitespace and capitalizing the first letter of the string. Validation ensures it is a non-empty string with a maximum length of 20 characters. If any condition fails during the process, the system generates an error message indicating the specific issue.

Sanitize

Validate

Extending Derive Functionality

If you need custom sanitizers or validators, you can extend the derive system by configuring the project to include your modules. Here's how:

Step 1: Add Custom Modules to Your Configuration

Application.put_env(:guarded_struct, :validate_derive, [TestValidate, TestValidate2])
Application.put_env(:guarded_struct, :sanitize_derive, [TestSanitize, TestSanitize2])

# OR
Application.put_env(:guarded_struct, :validate_derive, TestValidate)
Application.put_env(:guarded_struct, :sanitize_derive, TestSanitize)

Step 2: Create the Custom Modules (Custom Validators)

defmodule TestValidateModule do
  def validate(:testv1, input, field) do
    if is_binary(input),
      do: input,
      else: {:error, field, :testv1, "The #{field} field must not be empty"}
  end
end

defmodule TestValidateModule2 do
  def validate(:testv2, input, field) do
    if is_binary(input),
      do: input,
      else: {:error, field, :testv1, "The #{field} field must not be empty"}
  end
end

Custom Sanitizers

defmodule TestSanitizeModuleOne do
  def sanitize(:capitalize_v1, input) do
    if is_binary(input), do: String.capitalize(input) <> "::_v1", else: input
  end
end

defmodule TestSanitizeModuleTwo do
  def sanitize(:capitalize_v2, input) do
    if is_binary(input), do: String.capitalize(input) <> "::_v2", else: input
  end
end

Key Considerations

The library distinguishes between Core Derives and Custom Derives to streamline functionality. Core derives prioritize built-in derive names, while custom derives require unique names to avoid conflicts with existing functionality. To enhance clarity and modularity, the library offers a declarative configuration approach through the derive attribute, simplifying the setup of both sanitizers and validators. When validation errors occur, they are reported in a standardized format, such as {:error, [%{message: "Not right!", field: :title, action: :validator}]} , ensuring consistency and ease of integration. Additionally, the derive system is designed for composability, enabling developers to combine sanitizers and validators flexibly, even for nested or complex data structures.

Summary

By using the derive concept in GuardedStruct, you gain a powerful and extensible system for sanitizing and validating your data. Whether you stick to the core features or extend it with custom logic, GuardedStruct enables a clean, declarative approach inspired by Rust while fully leveraging Elixir’s flexibility.

Can You Combine Derive with Custom Validators?

Yes, you can combine derive with custom validators in GuardedStruct. The system prioritizes validators first and only applies derive rules once the data passes all validator checks. This layered approach ensures robust validation while allowing you to benefit from the declarative power of derive for sanitization and additional validation.

Example: Combining Derive and Custom Validators

defmodule NormalSimultaneouslyDeriveAndValidatorStruct do
  use GuardedStruct

  guardedstruct do
    field(:name, String.t(),
      enforce: true,
      derive: "sanitize(trim, upcase) validate(not_empty)"
    )

    field(:title, String.t(), derive: "sanitize(trim, capitalize) validate(not_empty)")
  end

  # Custom validator for the :name field
  def validator(:name, value) do
    if is_binary(value), do: {:ok, :name, "Mishka   "}, else: {:error, :name, "No, never"}
  end

  # Default validator
  def validator(name, value) do
    {:ok, name, value}
  end
end

# Valid Input:
NormalSimultaneouslyDeriveAndValidatorStruct.builder(%{name: "mishka", title: "developer"})

# Result:
%NormalSimultaneouslyDeriveAndValidatorStruct{
  name: "MISHKA",
  title: "Developer"
}

# Invalid Input:
NormalSimultaneouslyDeriveAndValidatorStruct.builder(%{name: 123, title: ""})

# Result:
{:error, [%{message: "No, never", field: :name, action: :validator}]}

Workflow Breakdown

The workflow for validation follows a structured process. For fields with a custom validator, such as :name, the custom validation function ( validator(:name, value) ) is executed first. If the custom validator succeeds, the sanitized and validated value is returned, for example: {:ok, :name, "Mishka "} . However, if it fails, an error is immediately returned, halting further processing, such as {:error, :name, "No, never"} .

Once the custom validator passes, the derive logic is applied to the value. For the :name field, the sanitization step trims whitespace and converts the string to uppercase, while the validation step ensures the string is not empty. The process follows a specific order: first, the custom validator is executed, and if it succeeds, the derive sanitizers and validators are applied.

For fields without a custom validator, like :title , only the derive sanitizers and validators are applied, simplifying the workflow for such cases. This structure ensures both flexibility and consistency in handling field-specific rules.

Workflow Breakdown

• Use custom validators for business-critical or complex logic.
• Leverage derive for consistent sanitization and simple validation rules.
• Keep the two systems modular for better maintainability and readability.

By combining derive and custom validators, GuardedStruct enables both flexibility and consistency, making it a powerful tool for handling complex data validation workflows.

Analyzing and Utilizing Complex Nested Structs with GuardedStruct

The GuardedStruct library enables developers to handle deeply nested and complex data structures efficiently, offering powerful tools to define, validate, and sanitize fields in a production-ready manner. Let’s break down the example of the NestedStruct and highlight how GuardedStruct simplifies the process.

Core Features of the Example

1. Handling Simple Fields

For fields like name, the process is straightforward:

• Compile-Time: Ensures the field is of type String.t() .

• Run-Time:

  • Enforces that the field is mandatory (enforce: true)
  • Sanitizes by trimming whitespace and converting the string to uppercase.
  • Validates that the field is not empty.
  • Example: For the input: name: "mishka" After processing, the output becomes: name: "Mishka"

2. Managing Nested Structures

The sub_field macro is used to define nested structures, like auth and role. Each nested field can have its own derive rules, validators, or be marked as mandatory.

Example: The auth field includes:

• A server field that must match a regex pattern ( ^[a-zA-Z][email protected]$ ).
• An identity_provider field that is sanitized (trimmed and converted to lowercase) and validated ( non-empty )

3. Flexible Validators

The validator function provides additional flexibility for fields requiring custom logic. Example: The nickname field in the profile struct is validated using a custom function.

def validator(:nickname, value) do
  if is_binary(value),
    do: {:ok, :nickname, value},
    else: {:error, :nickname, "Invalid nickname"}
end

4. Complex Derive Rules

Fields can include multiple sanitization and validation rules within a single derive statement. For example:

derive: "sanitize(strip_tags, trim, capitalize) validate(string, not_empty, max_len=20, min_len=3)"

• Sanitize:

  • Removes HTML tags (strip_tags).
  • Trims whitespace.
  • Capitalizes the string.
• Validate: Ensures the value is a string, non-empty, and within the specified length range.

Example Input

# Processed Output -- Valid Data:
{:ok,
%NestedStruct{
  username: "Mishka",
  profile: %NestedStruct.Profile{
    nickname: "mishka",
    site: "https://elixir-lang.org"
  },
  auth: %NestedStruct.Auth{
    last_activity: "2023-08-20 16:54:07.841434Z",
    role: %NestedStruct.Auth.Role{
      action: "true",
      name: :user,
      status: %{status: 2}
    },
    identity_provider: "google",
    server: "[email protected]"
  },
  age: 18,
  family: "Group",
  name: "Mishka"
}}

# Error Output: If any validation fails, the error is returned in a structured format:
{:error,
[
  %{field: :profile, errors: [%{message: "Invalid nickname", field: :nickname}]},
  %{
    field: :auth,
    errors: [
      %{
        message: "Invalid DateTime format in the last_activity field",
        field: :last_activity,
        action: :datetime
      },
      %{
        field: :role,
        errors: [
          %{
            message: "Invalid boolean format in the action field",
            field: :action,
            action: :string_boolean
          }
        ]
      }
    ]
  }
]}

Benefits of GuardedStruct

• Saves Time: Define, validate, and sanitize complex structs in under 20 minutes, ready for production use.
• Simplifies Complexity:: Nested structures and dependencies are handled seamlessly, with clear separation of concerns.
• Error Transparency: Errors are returned in a structured format, making debugging easier.
• Production-Ready: With its layered validation and sanitization, GuardedStruct reduces risks and improves data consistency in APIs.

Conclusion

With GuardedStruct, you can effortlessly handle deeply nested data structures, apply robust validation rules, and enforce consistent data policies. The combination of declarative macros, custom validators, and extensible derive features makes it a powerful tool for building production-grade APIs in Elixir.

Does It Look Amazing to You?

Isn’t it beautiful? With so many fields, defining a struct alone would typically take at least 5 minutes, and setting up dependencies could take several hours. Yet, with GuardedStruct, you can accomplish all of this—including preparing it for production—in less than 20 minutes.

But the good news doesn’t stop here. In the following sections, we’ll introduce even more key features that can reduce tasks that usually take days to just under an hour.

Can Error Handling in GuardedStruct Be Combined with defexception ?

Yes, you can simply add the error: true parameter to the parent macro ( guardedstruct ), and during compile-time, the macro automatically creates a module for handling errors in the same module where it is defined. If your macro includes sub_field or nested fields, a separate module is created for each nested section.

For example:

• TestCallNestedStructWithError.Error
• TestCallNestedStructWithError.Auth.Error
• TestCallNestedStructWithError.Auth.Path.Error

Here’s the sample code:

defmodule TestCallNestedStructWithError do
  use GuardedStruct

  guardedstruct error: true do
    field(:name, String.t(), derive: "validate(string)")

    sub_field(:auth, struct(), error: true) do
      field(:action, String.t(), derive: "validate(not_empty)")

      sub_field(:path, struct(), error: true) do
        field(:name, String.t())
      end
    end
  end
end

Have You Wondered Why a Module Is Created for Each sub_field ?

This is one of the key features that makes nested structs highly reusable across different parts of your application. Imagine you want to use only the struct for the path field (which itself contains multiple keys and children) in another part of your app. Instead of using the entire global builder , you can create separate builders for each nested field.

Isn’t that amazing? It maximizes your productivity by allowing each nested section to be independently modularized and reusable in its own context.

How Can You Enforce Maximum Limitation on User Input?

Let’s explore another feature of this library: the authorized_fields option. This option lets you explicitly limit the fields a user can input, adding an extra layer of control. Additionally, we’ll compare this feature with enforce to clarify their differences.

If this option is not enabled, any undefined fields will be automatically ignored by the program. However, when this option is active, it will return an error if the user includes a field that is not in the specified list. Keep in mind that this behavior differs from required_fields, as validation for mandatory fields occurs only after this step.

Security Note: To ensure the correctness of this functionality, you must accurately specify each field name as a map key. For instance, in frameworks like Phoenix, additional care should be taken to validate field names, as neglecting this step could lead to security vulnerabilities.

Example

defmodule TestAuthorizeKeys do
  use GuardedStruct

  guardedstruct authorized_fields: true do
    field(:name, String.t(), derive: "validate(string)")

    sub_field(:auth, struct(), authorized_fields: true) do
      field(:action, String.t(), derive: "validate(not_empty)")

      sub_field(:path, struct()) do
        field(:name, String.t())
      end
    end
  end
end

Calling Structs from Other Modules

In GuardedStruct, we've made sure that large projects with more modularized codebases can easily use this library without any limitations. With this feature, you can call and reuse fields from another module, providing greater flexibility in managing data flow across your project.

Example Code: Reusing Structs Across Modules

defmodule ExternalAuthModuleStruct do
  use GuardedStruct

  guardedstruct do
    field(:action, String.t(), derive: "validate(not_empty)")
  end
end

defmodule ExternalAuthCallModuleStruct do
  use GuardedStruct

  guardedstruct do
    field(:name, String.t(), derive: "validate(string)")
    field(:auth_path, struct(), struct: ExternalAuthModuleStruct)
    field(:auth_path1, struct(), structs: ExternalAuthModuleStruct)
  end
end

In this example:

• The ExternalAuthCallModuleStruct module uses the struct defined in ExternalAuthModuleStruct for the auth_path and auth_path1 fields.
• The auth_path field accepts a single map as input.
• The auth_path1 field accepts a list of maps as input

Input and Builder

Using this setup, the input format and builder function remain straightforward:

ExternalAuthCallModuleStruct.builder(%{
  name: "Mishka",
  auth_path: %{action: "add"},
  auth_path1: [%{action: "add"}]
})

# Result:

%ExternalAuthCallModuleStruct{
  name: "Mishka",
  auth_path: %ExternalAuthModuleStruct{action: "add"},
  auth_path1: [%ExternalAuthModuleStruct{action: "add"}]
}

Key Benefits

• Powerful Customization:: This feature allows for seamless customization of the data flow management in your project.
• Enhanced Modularity:: Structs from one module can be reused in others, promoting code reuse and reducing duplication.
• Support for Lists and Maps:: Even if the external module manages only maps, you can handle a list of data without modifying the original module.

Example: Handling Lists with Map Structs

Let’s look at how you can pass a list of data to a struct that only supports maps, without altering the external module:

ExternalAuthCallModuleStruct.builder(%{
  name: "Mishka",
  auth_path: %{action: "add"},
  auth_path1: [%{action: "add"}, %{action: "remove"}]
})

This flexibility means you can adapt the struct behavior to your specific needs without changing the core logic of the external module being called.

Conclusion

This feature provides immense flexibility for large-scale projects, allowing for the modularization and reuse of structs across different modules. It simplifies the management of nested and external data while ensuring seamless integration with GuardedStruct's builder functionality.

Example: Nested Fields with Lists and External Structs

In GuardedStruct, you can handle more complex scenarios, such as lists of nested structs or combining external structs with nested fields. Here’s an example that demonstrates the flexibility of this approach:

defmodule ListSubFieldAndExternalAuthCallModuleStruct do
  use GuardedStruct

  guardedstruct do
    field(:name, String.t(), derive: "validate(not_empty)")
    field(:auth_path, struct(), structs: ExternalAuthModuleStruct)

    sub_field(:profile, list(struct()), structs: true) do
      field(:github, String.t(), enforce: true, derive: "validate(url)")
      field(:nickname, String.t(), derive: "validate(not_empty)")
    end
  end
end

Example Input

Here’s how you can handle this struct in your API or function inputs:

# Input Example 1:
ListSubFieldAndExternalAuthCallModuleStruct.builder(%{
  name: "mishka",
  auth_path: [
    %{action: "*:admin", path: %{role: "1"}},
    %{action: "*:user", path: %{role: "3"}}
  ]
})

# Input Example 2:
ListSubFieldAndExternalAuthCallModuleStruct.builder(%{
  name: "mishka",
  auth_path: [
    %{action: "*:admin", path: %{role: "1"}},
    %{action: "*:user", path: %{role: "3", rel: %{social: "github"}}}
  ],
  profile: [%{github: "https://github.com/mishka-group"}]
})

Why Is This So Exciting?

I often find myself thrilled when working with the simplicity and power of the features I’ve built into this library. Starting a project or implementing its logic used to feel overwhelming and time-consuming. But now, with all the capabilities mentioned, there’s no excuse for delaying the start of a new project or adding a new feature.

What’s Next?

We haven’t even touched on some of the more advanced and unique features of this project yet. I guarantee that as we delve deeper, you’ll start imagining countless possibilities and implementations. You can explore the entire feature set in the documentation. While this article aims to cover the highlights, it’s impossible to dive deeply into every feature in one place. For detailed exploration, the documentation is your best resource.

Can a Field Be Automatically Revalued?

Yes, GuardedStruct introduces the Auto Core Key feature, which goes beyond the standard default functionality. Unlike default, which assigns a value when the user omits a field, Auto Core Key disregards the user's input entirely—even if the field is included in the input—and assigns a value based on a custom function.

For example, you may want an ID field to always generate a random string, regardless of whether the user provides an input. This feature is especially useful for ensuring consistent data generation.

Example: Auto Core Key Implementation

defmodule AutoCoreKeyStruct do
  use GuardedStruct

  guardedstruct do
    field(:username, String.t(), derive: "validate(not_empty)")
    field(:user_id, String.t(), auto: {Ecto.UUID, :generate})
    field(:parent_id, String.t(), auto: {Ecto.UUID, :generate})

    sub_field(:profile, struct()) do
      field(:id, String.t(), auto: {Ecto.UUID, :generate})
      field(:nickname, String.t(), derive: "validate(not_empty)")

      sub_field(:social, struct()) do
        field(:id, String.t(), auto: {AutoCoreKeyStruct, :create_uuid, "test-path"})
        field(:skype, String.t(), derive: "validate(string)")
        field(:username, String.t(), from: "root::username")
      end
    end

    sub_field(:items, struct(), structs: true) do
      field(:id, String.t(), auto: {Ecto.UUID, :generate})
      field(:something, String.t(), derive: "validate(string)", from: "root::username")
    end
  end

  # Custom UUID generator
  def create_uuid(default) do
    MishkaDeveloperTools.Helper.UUID.generate() <> "-#{default}"
  end
end

How Does It Work?

• Automatic Value Assignment: Fields marked with auto will always assign a value based on the provided function, regardless of user input.

  field(:user_id, String.t(), auto: {Ecto.UUID, :generate})

• Handling Nested Fields: The auto option works seamlessly within sub_field and nested structs.

  You can even pass arguments to the custom function, as seen in the social field:

  field(:id, String.t(), auto: {AutoCoreKeyStruct, :create_uuid, "test-path"})

• Flexible Input Handling: For custom operations like editing forms, you can override the auto behavior based on the builder's tag, such as :add or :edit .

Example Input and Output

# Example 1: Auto Value Ignored for user_id

AutoCoreKeyStruct.builder(%{username: "mishka", user_id: "test_not_to_be_replaced"})

# Result:

%AutoCoreKeyStruct{
  username: "mishka",
  user_id: "generated-uuid",
  parent_id: "generated-uuid",
  ...
}

# Example 2: Auto Value Considered for Editing (:edit)

AutoCoreKeyStruct.builder({:root, %{username: "mishka", user_id: "test_should_be_replaced"}, :edit})

# Result:
%AutoCoreKeyStruct{
  username: "mishka",
  user_id: "test_should_be_replaced",
  parent_id: "generated-uuid",
  ...
}

Key Features of Auto Core Key

• Custom Logic for Auto Values: Define specific rules or generation logic using external modules or functions.
• Flexible Use Casess: Works for both top-level fields and nested fields in complex structs.
• Override Capability: Allows exceptions for specific operations, such as editing, to accept user-provided values.
• Enhances Data Consistency: Ensures that sensitive fields like ID are always system-controlled and not user-defined.

Why Is This Important?

This feature not only streamlines data handling but also minimizes potential security risks by removing reliance on user-provided inputs for critical fields. The ability to customize and override behavior for specific operations (e.g., :edit ) adds a layer of flexibility that is crucial for real-world applications.

If you're exploring this feature further, remember that this is just one of many powerful options available in GuardedStruct. For a full list of capabilities, refer to the project documentation (click here ).

How Can You Create Dependent Fields?

One way to achieve dependent fields is by using the features we’ve discussed so far, such as custom validators and other tools. However, GuardedStruct introduces a specific concept called On Core Key, designed to make this process simpler and more declarative.

What Is the On Core Key Concept?

Using On Core Key, you can make one field dependent on the presence or value of another field. If the dependency isn’t met, an error message is generated. This allows you to easily enforce relationships between fields in your data structures.

How Does Routing Work?

When defining dependencies with On Core Key, the routing of data behaves differently depending on the configuration:

• When :root Is Specified: Routing starts from the root of the map provided as input.
• When :root Is Not Used: Routing starts from the child map received during processing.

Trade-Off of Using On Core Key

Using this feature comes with a trade-off. When core keys are called, the entire primary map is sent to each child for processing. This can increase memory usage depending on the size of the map, so it should be used thoughtfully in performance-critical scenarios.

defmodule OnCoreKeyStruct do
  use GuardedStruct

  guardedstruct do
    field(:name, String.t(), derive: "validate(string)")

    sub_field(:profile, struct()) do
      field(:id, String.t(), auto: {Ecto.UUID, :generate})
      field(:nickname, String.t(), on: "root::name", derive: "validate(string)")
      field(:github, String.t(), derive: "validate(string)")

      sub_field(:identity, struct()) do
        field(:provider, String.t(), on: "root::profile::github", derive: "validate(string)")
        field(:id, String.t(), auto: {Ecto.UUID, :generate})
        field(:rel, String.t(), on: "sub_identity::auth_path::action")

        sub_field(:sub_identity, struct()) do
          field(:id, String.t(), auto: {Ecto.UUID, :generate})
          field(:auth_path, struct(), struct: ExternalAuthModuleStruct)
        end
      end
    end

    sub_field(:last_activity, list(struct()), structs: true) do
      field(:action, String.t(), enforce: true, derive: "validate(string)", on: "root::name")
    end
  end
end

Now you can test like this:

OnCoreKeyStruct.builder(%{
  name: "mishka",
  profile: %{
    nickname: "Mishka",
    github: "test",
    identity: %{
      provider: "git",
      sub_identity: %{id: "test", auth_path: %{action: "admin/edit"}}
    }
  }
})

# OR

OnCoreKeyStruct.builder(%{
  name: "mishka",
  profile: %{
    nickname: "Mishka",
    identity: %{provider: "git"}
  }
})

It's just as simple to navigate between sub_fields and implement your logic as needed.

How to Retrieve the Value from Another Field

You might wonder why such a feature is necessary. The reason lies in the complex payloads you'll encounter in a full-fledged project using ActivityPub. This capability helps us build a complete, high-performance project tailored to various requirements.

Note: I must emphasize once again to refer to the documentation linked at the beginning of this article. Each section comes with specific rules and requirements that may not be fully discussed here.

defmodule FromCoreKeyStruct do
  use GuardedStruct

  guardedstruct do
    field(:username, String.t(), derive: "validate(not_empty)")
    field(:user_id, String.t(), auto: {Ecto.UUID, :generate})
    field(:parent_id, String.t(), auto: {Ecto.UUID, :generate})

    sub_field(:profile, struct()) do
      field(:id, String.t(), auto: {Ecto.UUID, :generate})
      field(:nickname, String.t(), derive: "validate(not_empty)")

      sub_field(:social, struct()) do
        field(:id, String.t(), auto: {FromCoreKeyStruct, :create_uuid, "test-path"})
        field(:skype, String.t(), derive: "validate(string)")
        field(:username, String.t(), from: "root::username")
      end
    end

    sub_field(:items, struct(), structs: true) do
      field(:id, String.t(), auto: {Ecto.UUID, :generate})
      field(:something, String.t(), derive: "validate(string)", from: "root::username")
    end
  end

  def create_uuid(default) do
    MishkaDeveloperTools.Helper.UUID.generate() <> "-#{default}"
  end
end

As we near the end of this article, you’ll notice that the complexity of the incoming data increases. This leads us to the next concept: Domain Core Key. Sometimes, you move beyond simple logic and into domain-level rules and abstractions.

What is Domain Core Key and How Does It Aggregate Data?

When working with deeply nested structures, there are times when you need to define the acceptable range of values for a set of parameters based on input from a parent. Unlike earlier features, in this section, we do not pass the entire Struct or Map into this feature. Instead, we rely on a top-down structure, often referred to as a parent-to-child relationship.

Example Code: Domain Core Key

defmodule AllowedParentDomain do
  use GuardedStruct

  guardedstruct authorized_fields: true do
    field(:username, String.t(),
      domain: "!auth.action=String[admin, user]::?auth.social=Atom[banned]",
      derive: "validate(string)"
    )

    field(:type_social, String.t(),
      domain: "?auth.type=Map[%{name: \"mishka\"}, %{name: \"mishka2\"}]",
      derive: "validate(string)"
    )

    sub_field(:auth, struct(), authorized_fields: true) do
      field(:action, String.t(), derive: "validate(not_empty)")
      field(:social, atom(), derive: "validate(atom)")
      field(:type, map(), derive: "validate(map)")
    end
  end
end

Explanation of Domain Core Key

1. Dependency Between Fields

The domain option allows you to enforce specific rules based on the values of related fields.

For instance:

domain: "!auth.action=String[admin, user]::?auth.social=Atom[banned]"

• ! indicates a required condition.
• ? indicates an optional condition.
• This rule enforces that auth.action must be either admin or user and that auth.social may optionally be banned .

2. Nested Relationships

The sub_field :auth defines additional rules and dependencies for action , social , and type .

3. Controlled Input

This approach ensures that child values align with parent-defined constraints, providing a robust way to handle deeply nested data.

Why Is This Feature Powerful?

The Domain Core Key feature allows you to create a pipeline for validating data at every level of a structure, ensuring consistency and reducing the chances of error. With its flexibility, you can handle a variety of data types and complex validation scenarios, making it a key tool for large-scale projects.

Note

This section is incredibly comprehensive, as you can define multiple domains, each with its own set of conditions. For example, the documentation dedicates 7-8 sections to this feature alone. Its high utility warrants even 1-2 dedicated tutorial videos for training purposes.

Final Feature: Conditional Fields

Are you excited yet? If not, you should be, because this feature goes far beyond a simple if-else. Each field can be:

• A Map.
• A List of Maps.
• An Empty String.
• A List of Strings.

And for each, you can define unique conditions. This allows you to create a pipeline for processing all user-submitted data in a single endpoint.

Benefits for Frontend Teams

This feature is also immensely helpful for frontend teams. It enables them to make better decisions when dealing with a large volume of errors, ensuring smoother workflows and better error management across the application.

What Are Conditional Fields?

One of the unique features of this macro is the ability to define conditional fields that adapt to different types of input. For example, you can configure a field such as social to accept either a string value or a map containing keys like address and provider.

Additionally, conditional_field supports nesting, allowing you to define subfields and complex hierarchies. Below, you'll find an example illustrating this flexibility.

Note on Documentation

When documenting your API's conditional fields, it's crucial to communicate the intended usage to your frontend team. Use the hint keyword to provide clues about how each section should behave. The output of hint will look like this: hint .

Example Code: Validators for Conditional Fields

Here’s an example of simple validators used for conditional fields:

defmodule ConditionalFieldValidatorTestValidators do
  def is_string_data(field, value) do
    if is_binary(value), do: {:ok, field, value}, else: {:error, field, "It is not a string"}
  end

  def is_map_data(field, value) do
    if is_map(value), do: {:ok, field, value}, else: {:error, field, "It is not a map"}
  end

  def is_list_data(field, value) do
    if is_list(value), do: {:ok, field, value}, else: {:error, field, "It is not a list"}
  end

  def is_flat_list_data(field, value) do
    if is_list(value),
      do: {:ok, field, List.flatten(value)},
      else: {:error, field, "It is not a list"}
  end

  def is_int_data(field, value) do
    if is_integer(value), do: {:ok, field, value}, else: {:error, field, "It is not an integer"}
  end
end

Using Conditional Fields in a Schema

The example below shows how to use the conditional_field macro to define complex nested fields:

defmodule ConditionalFieldComplexTest do
  use GuardedStruct
  alias ConditionalFieldValidatorTestValidators, as: VAL

  guardedstruct do
    field(:provider, String.t())

    sub_field(:profile, struct()) do
      field(:name, String.t(), enforce: true)
      field(:family, String.t(), enforce: true)

      conditional_field(:address, any()) do
        field(:address, String.t(), hint: "address1", validator: {VAL, :is_string_data})

        sub_field(:address, struct(), hint: "address2", validator: {VAL, :is_map_data}) do
          field(:location, String.t(), enforce: true)
          field(:text_location, String.t(), enforce: true)
        end

        sub_field(:address, struct(), hint: "address3", validator: {VAL, :is_map_data}) do
          field(:location, String.t(), enforce: true, derive: "validate(string, location)")
          field(:text_location, String.t(), enforce: true)
          field(:email, String.t(), enforce: true)
        end
      end
    end

    conditional_field(:product, any()) do
      field(:product, String.t(), hint: "product1", validator: {VAL, :is_string_data})

      sub_field(:product, struct(), hint: "product2", validator: {VAL, :is_map_data}) do
        field(:name, String.t(), enforce: true)
        field(:price, integer(), enforce: true)

        sub_field(:information, struct()) do
          field(:creator, String.t(), enforce: true)
          field(:company, String.t(), enforce: true)

          conditional_field(:inventory, integer() | struct(), enforce: true) do
            field(:inventory, integer(),
              hint: "inventory1",
              validator: {VAL, :is_int_data},
              derive: "validate(integer, max_len=33)"
            )

            sub_field(:inventory, struct(), hint: "inventory2", validator: {VAL, :is_map_data}) do
              field(:count, integer(), enforce: true)
              field(:expiration, integer(), enforce: true)
            end
          end
        end
      end
    end
  end
end

Example Input Payload

Here’s a sample payload a user can send to the builder function:

ConditionalFieldComplexTest.builder(%{
  provider: "Mishka",
  profile: %{
    name: "Shahryar",
    family: "Tavakkoli",
    address: %{
      location: "geo:48.198634,-16.371648,3.4;crs=wgs84;u=40.0",
      text_location: "Nowhere",
      email: "[email protected]"
    }
  },
  product: %{
    name: "MishkaDeveloperTools",
    price: 0,
    information: %{
      creator: "Shahryar Tavakkoli",
      company: "Mishka Group",
      inventory: %{
        count: 3_000_000,
        expiration: 33
      }
    }
  }
})

Flexibility of Conditional Fields

If you review the code, you’ll notice that you can use both validator and derive with conditional fields. This feature’s flexibility allows for complex configurations. For example, by changing just one s, you can switch a single conditional field into a list of conditional fields:

conditional_field(:activities, any(), structs: true) do
  field(:activities, struct(), struct: ExtrenalConditional, validator: {VAL, :is_map_data}, hint: "activities1")
  field(:activities, struct(), structs: ExtrenalConditional, validator: {VAL, :is_list_data}, hint: "activities2")
  field(:activities, String.t(), hint: "activities3", validator: {VAL, :is_string_data})
end

Summary

This schema outlines the extensive capabilities of this macro. While developing and testing, I encountered a problem I couldn’t solve. If you think you can help, I’ve opened an issue for it.

I’m planning to create short videos explaining each feature soon and will share them with you!

If you found this tutorial helpful, please share it on social media!