Determine specific point data on each layer in the stack using rasterVis in R

I have two rasters stacked together:

library(rasterVis)
r1 <- raster(system.file("external/test.grd", package="raster")) 
r2 <- r1 / 2
r.stack <- stack(r1, r2)

      

        
        
        
      

    

Since I would like to highlight some areas later for each specific layer in the stack, I create two point datasets based on the raster values:

pts1 <- rasterToPoints(r1, spatial=T)
idx <- which(as.data.frame(pts1)[, 1] >= 400)
pts1 <- pts1[idx, 1]

pts2 <- rasterToPoints(r2, spatial=T)
idx <- which(as.data.frame(pts2)[, 1] >= 400)
pts2 <- pts2[idx, 1]

      

        
        
        
      

    

Now I would like to build a raster stack with levelplot from rasterVis to R. I would also like to overlay r1 on pts1 and r2 using pts2.

However, once I add one point dataset, it is used for both layers:

levelplot(r.stack) + layer(sp.points(pts1, pch=20, cex=0.1, col="black"))

      

        
        
        
      

    

How can I use specific point datasets with specific layers when using raster stacks?

I would like to avoid creating my own subplot by building each layer with its "specific dataset" and then using print.trellis. I tried it, but the result was slightly worse compared to levelplot with raster stacks.

Any idea on how to achieve this?