Apache Beam returns "Input values must not be mutated in any way." when using local straight runner
I wrote Apache Beam DoFn
static class FillLocation extends DoFn<TrackingRequest, TrackingRequest> {
@ProcessElement
public void processElement(ProcessContext c) {
TrackingRequest rq = c.element();
rq.location = getLocationFromIP(rq.IP);
c.output(rq);
}
}
And it gave me this error when testing PTransform locally .. illegal mutated value of .. class .....
Input values must not be mutated in any way.
at org.apache.beam.runners.direct.ImmutabilityEnforcementFactory$ImmutabilityCheckingEnforcement.verifyUnmodified(ImmutabilityEnforcementFactory.java:96)
at org.apache.beam.runners.direct.ImmutabilityEnforcementFactory$ImmutabilityCheckingEnforcement.afterElement(ImmutabilityEnforcementFactory.java:71)
at org.apache.beam.runners.direct.TransformExecutor.processElements(TransformExecutor.java:149)
at org.apache.beam.runners.direct.TransformExecutor.run(TransformExecutor.java:107)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
+3
source to share
1 answer
Your function changed the location field of the TrackingRequest input element. This is not permitted in the data stream.
doc says:
The current PCollection input element is returned by c.element (). It should be considered immutable. The Dataflow runtime will not mutate the item, so it is safe for caching, etc. The element must not be mutated by any of the DoFn methods, as it can be cached elsewhere, saved at runtime Dataflow, or used in other undefined ways.
You can create a copy of the input element, change the field, and send the copy as output.
+4
source to share