#Function
In PySpark, when().otherwise() is used for conditional column expressions, similar to SQL CASE WHEN.
Syntax: -
from pyspark.sql.functions import when
new_col = when(condition, value_if_true).otherwise(value_if_false)
Example: -
df = df.withColumn("new_col", when(col("age") > 18, "Adult").otherwise("Minor"))
Can be used to update existing columns, handling null values etc.
# Apply multiple conditions
df_with_conditions = df.withColumn(
"category_label",
when(col("category") == "A", "Label 1")
.when(col("category") == "B", "Label 2")
.when(col("category") == "C", "Label 3")
.otherwise("Other")
)