grouped_by fails if the foreign key uses #[diesel(serialize_as, deserialize_as)]

[silmeth]

Rust Version

1.93.0

Diesel Version

2.3.6

Diesel Features

sqlite

Database Version

Sqlite 3.46.1

Operating System Version

Kubuntu 25.10

What happened?

When using joinable tables whose struct definitions use field types mapped using #[diesel(serialize_as)] and #[diesel(deserialize_as)] in their primary and foreign key fields, their relation is not usable with the GroupBy trait, the compiler complaining, among others, that [ChildEntity] is required to implement Sized for Vec<ChildEntity> to implement GroupBy.

While most often one uses simple types like i64, i32, or sometimes String for table keys, mapping them via separate mapping types comes in handy if one wants eg. Ulid keys serialized as raw bytes in the database (that’s actually how I first hit the problem).

Using the conversion type directly (instead of via serialize_as / deserialize_as attributes) in the struct works around this limitation.

What did you expect to happen?

I expected either:

1. the grouped_by function to work in this case,
2. or the limitation (if it is intended eg. to avoid issues with different behaviour of Hash and Eq/PartialEq on the two types) to be documented and the error message clearer.

I lost a few hours trying to figure out why this doesn’t want to work and why it requires an unsized array to implement sized.

Additional details

I checked that this issue can be reproduced without a third party crate as it’s true – for brevity I pulled derive_more in the minimal reproducer though.

Steps to reproduce

A full reproducible example using these dependencies:

derive_more = { version = "2.1.1", features = ["from", "into"] }
diesel = { version = "2.3.6", features = ["sqlite"] }

is here:

main.rs

use derive_more::{From, Into};
use diesel::{
    backend::Backend,
    deserialize::{FromSql, FromSqlRow},
    expression::AsExpression,
    prelude::*,
    serialize::{IsNull, ToSql},
    sql_types,
    sqlite::Sqlite,
};

mod schema {
    diesel::table! {
        posts (id) {
            id -> Text,
            content -> Text,
        }
    }

    diesel::table! {
        tags_to_posts (name, post_id) {
            name -> Text,
            post_id -> Text,
        }
    }

    diesel::allow_tables_to_appear_in_same_query!(posts, tags_to_posts,);
}

#[derive(Debug, PartialEq, Eq, Hash, Selectable, Queryable, Identifiable, Insertable)]
#[diesel(table_name = schema::posts)]
struct Post {
    #[diesel(deserialize_as = IntAsString, serialize_as = IntAsString)]
    id: i32,
    content: String,
}

#[derive(
    Debug, PartialEq, Eq, Hash, Selectable, Queryable, Identifiable, Insertable, Associations,
)]
#[diesel(table_name = schema::tags_to_posts)]
#[diesel(primary_key(name, post_id))]
#[diesel(belongs_to(Post))]
struct TagToPost {
    name: String,
    #[diesel(deserialize_as = IntAsString, serialize_as = IntAsString)]
    post_id: i32,
}

#[derive(Debug, PartialEq, Eq, Hash, From, Into, AsExpression, FromSqlRow)]
#[diesel(sql_type = sql_types::Text)]
struct IntAsString(i32);

impl FromSql<sql_types::Text, Sqlite> for IntAsString {
    fn from_sql(bytes: <Sqlite as Backend>::RawValue<'_>) -> diesel::deserialize::Result<Self> {
        let string = <String as FromSql<sql_types::Text, _>>::from_sql(bytes)?;
        let i = string.parse().map_err(Box::new)?;
        Ok(IntAsString(i))
    }
}

impl ToSql<sql_types::Text, Sqlite> for IntAsString {
    fn to_sql<'b>(
        &'b self,
        out: &mut diesel::serialize::Output<'b, '_, Sqlite>,
    ) -> diesel::serialize::Result {
        let string = self.0.to_string();
        out.set_value(string);
        Ok(IsNull::No)
    }
}

fn main() {
    let mut db = diesel::SqliteConnection::establish("test.db").unwrap();
    let posts = schema::posts::table
        .select(Post::as_select())
        .load::<Post>(&mut db)
        .unwrap();
    let tags = schema::tags_to_posts::table
        .select(TagToPost::as_select())
        .load::<TagToPost>(&mut db)
        .unwrap();

    let _grouped = tags
        .grouped_by(&posts)
        .into_iter()
        .zip(posts)
        .collect::<Vec<_>>();
}

Compile time error

error[E0599]: the method `grouped_by` exists for struct `Vec<TagToPost>`, but its trait bounds were not satisfied
  --> src/main.rs:85:10
   |
84 |       let _grouped = tags
   |  ____________________-
85 | |         .grouped_by(&posts)
   | |         -^^^^^^^^^^ method cannot be called on `Vec<TagToPost>` due to unsatisfied trait bounds
   | |_________|
   |
   |
   = note: the following trait bounds were not satisfied:
           `<[TagToPost] as IntoIterator>::Item = _`
           which is required by `[TagToPost]: diesel::GroupedBy<'_, _>`
           `[TagToPost]: Sized`
           which is required by `[TagToPost]: diesel::GroupedBy<'_, _>`
           `[TagToPost]: IntoIterator`
           which is required by `[TagToPost]: diesel::GroupedBy<'_, _>`

For more information about this error, try `rustc --explain E0599`.
error: could not compile `diesel-associations` (bin "diesel-associations") due to 1 previous error

Checklist

• I have already looked over the issue tracker and the discussion forum for similar possible closed issues.

• This issue can be reproduced on Rust's stable channel. (Please submit the issue in the Rust issue tracker instead if that's not the case)

• This issue can be reproduced without requiring a third party crate

[weiznich]

Thanks for reporting this issue. I think the compiler error message is definitivly bad and should be improved by putting #[diagnostic::do_not_recommend] and #[diagnostic::on_unimplemented] on the right item.

As for why this does not compile: It seems like the #[derive(Associations)] proc macro does not handle the #[diesel(serialize_as)] attribute correctly that causes a mismatch in the generated code between what #[derive(Identifiable)] expects as primary key type and what is actually generated, which in turn result in the relevant trait impl not applying.

The problematic place is here:

diesel/diesel_derives/src/associations.rs

Line 59 in 17c6352

parse_quote!(for<'__a> &'__a #foreign_key_ty: std::convert::Into<::std::option::Option<&'__a __FK>>),

The foreign key type there is currently generated as i32 while the Identifiable trait expects IntAsString type. We would need to do something like Identifiable does there as well:

diesel/diesel_derives/src/identifiable.rs

Lines 32 to 38 in 17c6352

	if let Some(AttributeSpanWrapper { item: ty, .. }) = serialize_as.as_ref() {
	field_ty.push(quote!(#ty));
	field_name.push(quote!(::std::convert::Into::<#ty>::into(self.#name.clone())));
	} else {
	field_ty.push(quote!(&'ident #ty));
	field_name.push(quote!(&self.#name));
	}

After fixing the code we also need to run at least the diesel-derives tests as they generate some snapshot of the generated code. Also we want to have a test case (the minimal example from the issue) as test for checks for future regressions.

Any help would be welcome.

[KevinKickass]

Hi @weiznich, I took a stab at implementing the fix you outlined but ran into a fundamental constraint with the current BelongsTo trait design. Wanted to share the analysis before going further.

The Problem with a Pure Derive Fix

The suggested fix (making associations.rs handle serialize_as like identifiable.rs does) doesn't fully work because of a lifetime/ownership mismatch:

1. Identifiable with serialize_as returns an owned value: Id = IntAsString (via Into::into(self.name.clone()))
2. BelongsTo::foreign_key() returns Option<&Self::ForeignKey> — a reference
3. If the struct field is i32 but ForeignKey needs to be IntAsString, we can't return &IntAsString — there's no IntAsString stored in the struct to reference

And the GroupedBy blanket impl requires:

Id<&'a Parent>: Borrow<Child::ForeignKey>

• If ForeignKey = IntAsString → need IntAsString: Borrow<IntAsString> ✓ but can't return &IntAsString from foreign_key()
• If ForeignKey = i32 (raw) → need IntAsString: Borrow<i32> ✗ (no such impl)

So no matter which type we choose for ForeignKey, one side of the equation breaks.

Proposed Solution: Cow Return Type

Change BelongsTo::foreign_key() to return Cow instead of a plain reference:

fn foreign_key(&self) -> Option<Cow<'_, Self::ForeignKey>>;

This allows the generated code to handle both cases:

With serialize_as (owned conversion):

impl BelongsTo<Post> for TagToPost {
    type ForeignKey = IntAsString;
    fn foreign_key(&self) -> Option<Cow<'_, IntAsString>> {
        Some(Cow::Owned(IntAsString::from(self.post_id.clone())))
    }
}

Without serialize_as (zero-cost borrow, like today):

impl BelongsTo<Post> for TagToPost {
    type ForeignKey = i32;
    fn foreign_key(&self) -> Option<Cow<'_, i32>> {
        Some(Cow::Borrowed(&self.post_id))
    }
}

The GroupedBy impl needs one small change:

let fk = child.foreign_key()?;       // Cow<'_, FK>
let i = id_indices.get(fk.as_ref())?;  // &FK

Why this works:

• Cow::Borrowed is zero-cost for the normal (non-serialize_as) path
• Cow::Owned handles the conversion case where no stored reference exists
• Cow<T> implements AsRef<T>, so the HashMap lookup works
• The Id: Borrow<ForeignKey> constraint is satisfied: IntAsString: Borrow<IntAsString> ✓
• Cow requires ToOwned, but that's blanket-implemented for all Clone types

What breaks:

• Manual BelongsTo implementations (must update return type to Cow)
• In practice, ~99% of users use #[derive(Associations)] and would be migrated automatically by the updated derive macro

What doesn't break:

• All derive-based code (gets Cow::Borrowed by default — same semantics as today)
• belonging_to() queries (uses foreign_key_column(), not foreign_key())
• Query building in general

Happy to implement this if you think the approach is sound. Would this be acceptable as a breaking change in a 2.x minor (given the limited manual-impl surface), or would it need to wait for 3.0?

[weiznich]

Thanks for looking into this. It's unfortunate that a fix would require a breaking change.
We don't want to perform such changes without major version bump, even if it only affects a very narrow set of users.
At this point we don't have any specific plans for a diesel 3.0 release, so this need to wait until then.

I will nevertheless try to have another look in the next weeks, maybe I have another idea how to work around that issue.