Details
-
Question
-
Status: Resolved
-
Critical
-
Resolution: Not A Problem
-
3.2.0
-
None
Description
I have an ALS model and saved it with these codes:
als_path = "saved_models/best"
best_model.save(sc, path= als_path)
However, when I try to load this model, it gives this error message:
---> 10 model2 = ALS.load(als_path) File /usr/local/spark/python/pyspark/ml/util.py:332, in MLReadable.load(cls, path) 329 @classmethod 330 def load(cls, path): 331 """Reads an ML instance from the input path, a shortcut of `read().load(path)`.""" --> 332 return cls.read().load(path) File /usr/local/spark/python/pyspark/ml/util.py:282, in JavaMLReader.load(self, path) 280 if not isinstance(path, str): 281 raise TypeError("path should be a string, got type %s" % type(path)) --> 282 java_obj = self._jread.load(path) 283 if not hasattr(self._clazz, "_from_java"): 284 raise NotImplementedError("This Java ML type cannot be loaded into Python currently: %r" 285 % self._clazz) File /usr/local/spark/python/lib/py4j-0.10.9.3-src.zip/py4j/java_gateway.py:1321, in JavaMember.__call__(self, *args) 1315 command = proto.CALL_COMMAND_NAME +\ 1316 self.command_header +\ 1317 args_command +\ 1318 proto.END_COMMAND_PART 1320 answer = self.gateway_client.send_command(command) -> 1321 return_value = get_return_value( 1322 answer, self.gateway_client, self.target_id, self.name) 1324 for temp_arg in temp_args: 1325 temp_arg._detach() File /usr/local/spark/python/pyspark/sql/utils.py:111, in capture_sql_exception.<locals>.deco(*a, **kw) 109 def deco(*a, **kw): 110 try: --> 111 return f(*a, **kw) 112 except py4j.protocol.Py4JJavaError as e: 113 converted = convert_exception(e.java_exception) File /usr/local/spark/python/lib/py4j-0.10.9.3-src.zip/py4j/protocol.py:326, in get_return_value(answer, gateway_client, target_id, name) 324 value = OUTPUT_CONVERTER[type](answer[2:], gateway_client) 325 if answer[1] == REFERENCE_TYPE: --> 326 raise Py4JJavaError( 327 "An error occurred while calling {0}{1}{2}.\n". 328 format(target_id, ".", name), value) 329 else: 330 raise Py4JError( 331 "An error occurred while calling {0}{1}{2}. Trace:\n{3}\n". 332 format(target_id, ".", name, value)) Py4JJavaError: An error occurred while calling o372.load. : org.json4s.MappingException: Did not find value which can be converted into java.lang.String at org.json4s.reflect.package$.fail(package.scala:53) at org.json4s.Extraction$.$anonfun$convert$2(Extraction.scala:881) at scala.Option.getOrElse(Option.scala:189) at org.json4s.Extraction$.convert(Extraction.scala:881) at org.json4s.Extraction$.$anonfun$extract$10(Extraction.scala:456) at org.json4s.Extraction$.$anonfun$customOrElse$1(Extraction.scala:780)
I both tried to use `ALS.load` or `ALSModel.load` as shown in the Apache spark documentation: