On Windows, unable to install Surprise with poetry due to Cythonizing error

[omar-sobhy]

Description

Trying to install the package with poetry on Windows (maybe Linux too? haven't tested it myself) gives an error.

Skimming through the error log, this seems to be an error because of the usage of np.int_t, which is no longer supported by numpy.

Steps/Code to Reproduce

Run poetry add scikit-surprise.

Expected Results

The package installs successfully.

Actual Results

Output from `poetry add scikit-surprise` command (long output)

> poetry add scikit-surprise                                       
Using version ^1.1.4 for scikit-surprise

Updating dependencies
Resolving dependencies... (0.6s)

Package operations: 1 install, 0 updates, 0 removals

  - Installing scikit-surprise (1.1.4): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke get_requires_for_build_wheel
  
  
  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          self.avg_cltr_i = avg_cltr_i
          self.avg_cocltr = avg_cocltr
  
          return self
  
      def compute_averages(self, np.ndarray[np.int_t] cltr_u,
                                               ^
  ------------------------------------------------------------
  
  surprise\prediction_algorithms\co_clustering.pyx:157:45: Invalid type.
  Compiling surprise/similarities.pyx because it changed.
  Compiling surprise/prediction_algorithms/matrix_factorization.pyx because it changed.
  Compiling surprise/prediction_algorithms/optimize_baselines.pyx because it changed.
  Compiling surprise/prediction_algorithms/slope_one.pyx because it changed.
  Compiling surprise/prediction_algorithms/co_clustering.pyx because it changed.
  [1/5] Cythonizing surprise/prediction_algorithms/co_clustering.pyx
  Traceback (most recent call last):
    File "C:\Users\Omar\pipx\venvs\poetry\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 389, in <module>
      main()
      ~~~~^^
    File "C:\Users\Omar\pipx\venvs\poetry\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 373, in main
      json_out["return_val"] = hook(**hook_input["kwargs"])
                               ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\Omar\pipx\venvs\poetry\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 143, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\Omar\AppData\Local\Temp\tmpnm79ijzm\.venv\Lib\site-packages\setuptools\build_meta.py", line 334, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=[])
             ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\Omar\AppData\Local\Temp\tmpnm79ijzm\.venv\Lib\site-packages\setuptools\build_meta.py", line 304, in _get_build_requires
      self.run_setup()
      ~~~~~~~~~~~~~~^^
    File "C:\Users\Omar\AppData\Local\Temp\tmpnm79ijzm\.venv\Lib\site-packages\setuptools\build_meta.py", line 320, in run_setup
      exec(code, locals())
      ~~~~^^^^^^^^^^^^^^^^
    File "<string>", line 116, in <module>
    File "C:\Users\Omar\AppData\Local\Temp\tmpnm79ijzm\.venv\Lib\site-packages\Cython\Build\Dependencies.py", line 1154, in cythonize
      cythonize_one(*args)
      ~~~~~~~~~~~~~^^^^^^^
    File "C:\Users\Omar\AppData\Local\Temp\tmpnm79ijzm\.venv\Lib\site-packages\Cython\Build\Dependencies.py", line 1321, in cythonize_one
      raise CompileError(None, pyx_file)
  Cython.Compiler.Errors.CompileError: surprise/prediction_algorithms/co_clustering.pyx
  

  at ~\pipx\venvs\poetry\Lib\site-packages\poetry\installation\chef.py:164 in _prepare
      160│
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│
      163│             if error is not None:
    → 164│                 raise error from None
      165│
      166│             return path
      167│
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with scikit-surprise (1.1.4) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "scikit-surprise (==1.1.4)"'.

Versions

Windows-11-10.0.22631-SP0
Python 3.13.0 (tags/v3.13.0:60403a5, Oct 7 2024, 09:38:07) [MSC v.1941 64 bit (AMD64)]

[shaibi007]

I have similar cython issue when installing using pip.

[NicolasHug]

Hi folks, you're probably using numpy 2 ? I haven't had a chance to add numpy 2 compatibility yet. Happy to get some help on this, in the meantime I think you'll have to use numpy < 2

[shaibi007]

I have tried downgrading numpy and reinstall surprise.
I get some errors with the installation:

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [45 lines of output]

  Error compiling Cython file:
  ------------------------------------------------------------
  ...
          self.avg_cltr_i = avg_cltr_i
          self.avg_cocltr = avg_cocltr

          return self

      def compute_averages(self, np.ndarray[np.int_t] cltr_u,
                                               ^
  ------------------------------------------------------------

  surprise\prediction_algorithms\co_clustering.pyx:157:45: Invalid type.
  Compiling surprise/similarities.pyx because it changed.
  Compiling surprise/prediction_algorithms/matrix_factorization.pyx because it changed.
  Compiling surprise/prediction_algorithms/optimize_baselines.pyx because it changed.
  Compiling surprise/prediction_algorithms/slope_one.pyx because it changed.
  Compiling surprise/prediction_algorithms/co_clustering.pyx because it changed.
  [1/5] Cythonizing surprise/prediction_algorithms/co_clustering.pyx
  Traceback (most recent call last):
    File "C:\Users\rshoaib\AppData\Local\Programs\Python\Python313\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
      main()
      ~~~~^^
    File "C:\Users\rshoaib\AppData\Local\Programs\Python\Python313\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\rshoaib\AppData\Local\Programs\Python\Python313\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "C:\Users\rshoaib\AppData\Local\Temp\pip-build-env-bihqyyo5\overlay\Lib\site-packages\setuptools\build_meta.py", line 334, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=[])
             ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\rshoaib\AppData\Local\Temp\pip-build-env-bihqyyo5\overlay\Lib\site-packages\setuptools\build_meta.py", line 304, in _get_build_requires
      self.run_setup()
      ~~~~~~~~~~~~~~^^
    File "C:\Users\rshoaib\AppData\Local\Temp\pip-build-env-bihqyyo5\overlay\Lib\site-packages\setuptools\build_meta.py", line 320, in run_setup
      exec(code, locals())
      ~~~~^^^^^^^^^^^^^^^^
    File "<string>", line 116, in <module>
    File "C:\Users\rshoaib\AppData\Local\Temp\pip-build-env-bihqyyo5\overlay\Lib\site-packages\Cython\Build\Dependencies.py", line 1154, in cythonize
      cythonize_one(*args)
      ~~~~~~~~~~~~~^^^^^^^
    File "C:\Users\rshoaib\AppData\Local\Temp\pip-build-env-bihqyyo5\overlay\Lib\site-packages\Cython\Build\Dependencies.py", line 1321, in cythonize_one
      raise CompileError(None, pyx_file)
  Cython.Compiler.Errors.CompileError: surprise/prediction_algorithms/co_clustering.pyx
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[CorentinDeBoisset]

I've had the same error, because I used a too recent version of python (3.13). Using pyton3.12 works fine.

[bgurkas]

Here's the fix for this problem (because I encountered the same thing!):

1. Go into surprise/prediction_algorithms/co_clustering.pyx
2. Replace every occurrence of int_t with int32_t or int64_t, depending on your precision preference.
3. Save and run python setup.py install from the main directory!

The issue arises because int_t is no longer supported in recent versions. This fixed it for me!

[RZed01]

Interesting. I've been unable to install surprise on python. I have tried 3.13.3, 3.12.10, and 3.8.10. None of them worked. No version worked either. 1.1.3, 1.1.4, 1.1.1, nothing worked. Older versions seem to want PEP 517, but even using that solves nothing. Using Windows 11.

[RZed01]

Main error with 1.1.4 on Python 3.8.10 is: "ndarray" is not a type identifier. Suspecting that the numpy version is too new: current version installed on that version of Python is 1.24.4.