How to use GGML_SYCL_F64?

[GermanAizek]

@ggerganov,
Hi again, I am a contributor to your project.
I have server GPU that has an advantage in FP64 tasks, is it possible to compile with such compile flag? Is there a way to rewrite llama.cpp for a larger data type?

[GermanAizek]

// partial sum for each thread
#ifdef GGML_SYCL_F16
    sycl::half2 tmp = {0.0f, 0.0f}; // two sums for f16 to take advantage of half2 intrinsics
#else
    float tmp = 0.0f;
#endif // GGML_SYCL_F16

judging by the code, I can increase all type sizes and append ifdef macros, but I need other contributors to double-check my commits, I may make a mistake.

[ggerganov]

It's not that easy. But regardless, we don't plan to support F64.

[GermanAizek]

It's not that easy. But regardless, we don't plan to support F64.

so you won't even accept PR?

[0cc4m]

What's your use case? 64-bit floats are mostly used for scientific simulation work, which is why AMD Instinct cards are common there. But ggml is a machine learning framework and I don't think I have heard of a machine learning advantage to 64-bit floats.

[qnixsynapse]

I don't know what's the use case here for supporting fp64? GGML is a Machine Learning framework. We don't have proper bf16 support yet in SYCL backend.

[GermanAizek]

64-bit floats are mostly used for scientific simulation work, which is why AMD Instinct cards are common there.

As if you knew what University was using, there really is a large cluster AMD Instinct GPU, and I noticed that many open source projects still do not have an FP64 use case, although this is a really convenient option for some hardware. Main question is, how much do I need to rewrite data types in code? I can do it, I can test it.