Protocop-net memcache provider null error type during deserialization

I am using latest protobuf-net lib with protobuf-net memcache provider. I need to serialize a list of custom type MyClass

[ProtoContract]
public class MyClass{
  [ProtoMember(1)]
  public int a {get; set;}
  [ProtoMember(2)]
  public int b {get; set;}
}

      

So I need to keep / retreive:

List<MyClass> myList

      

When the values โ€‹โ€‹are stored through protobuf and then returned from the cache, everything goes well. But if the value is stored in memcache (in the correct protobuf format), that is, from another thread / application, and after that this value is returned from cache deserialization due to NullReferenceExceptions of the type field.

So, when first is set, the type of the serialized value is stored in the typeCache variable and then returned from that dictionary. But if the value is present in memcache, but not set in the current thread, the typeCache var does not contain this type and throws a NullReference on deserialization.

Is there a way to fix this or some workaround?

Deeper Investigation: The serialization / deserialization process for enyim implemented in ProtoTranscoder.cs

. It contains a class NetTranscoder

that has Dictionary<ArraySegment<byte>, Type> typeCache

. So when set, the first serial type (i.e. List<MyClass>

) is stored in typeCache

var and everything goes well. If the value is present in memcache, but not set in the current application / thread during deserialization, it was restored with this code:

type = Type.GetType(enc.GetString(buffer, keyOffset, len));
byte[] standaloneBuffer = new byte[len];
Buffer.BlockCopy(buffer, keyOffset, standaloneBuffer, 0, len);
key = new ArraySegment<byte>(standaloneBuffer, 0, len);
sync.EnterWriteLock();
try
   {
        // did somebody beat us to it?
        Type tmp;
        if (typeCache.TryGetValue(key, out tmp)) return tmp;
        typeCache.Add(key, type);
        return type;   <-- Here it returns null, if type not present in typeCache
   }
finally
   {
        sync.ExitWriteLock();
   }

      

To reproduce this error:

  • List item
  • Create and save some list in memcache (with prototranscoder configured)
  • Restart the current application (or start another thread)
  • Try to get the value using the key from memcache from this "other thread"

Here is the stack trace of this error: [ArgumentNullException: The value cannot be null. Parameter name: type]

   ProtoBuf.Meta.TypeModel.PrepareDeserialize(Object value, Type& type) in c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:592
   ProtoBuf.Meta.TypeModel.Deserialize(Stream source, Object value, Type type, SerializationContext context) in c:\Dev\protobuf-net\protobuf-net\Meta\TypeModel.cs:577
   ProtoBuf.Caching.Enyim.NetTranscoder.Enyim.Caching.Memcached.ITranscoder.Deserialize(CacheItem item) in c:\Users\akureniov\work\protobuf-net-1\protobuf-net.Enyim\protobuf-net.Enyim\ProtoTranscoder.cs:109
   Enyim.Caching.MemcachedClient.PerformTryGet(String key, UInt64& cas, Object& value) +179
   Enyim.Caching.MemcachedClient.TryGet(String key, Object& value) +42
   Enyim.Caching.MemcachedClient.Get(String key) +15

      

+3


source to share


2 answers


With the help of Mark Gravell, we discovered a "bug" in type truncation in ProtoTranscoder.cs: NetTranscoder: void WriteType (MemoryStream ms, Type Type). This code block throws an error because it shortens too much:

int i = typeName.IndexOf(','); // first split
if (i >= 0) { i = typeName.IndexOf(',', i + 1); } // second split
if (i >= 0) { typeName = typeName.Substring(0, i); } // extract type/assembly only

      

It worked well for simple types, but doesn't work for List, Dictionary, etc.

To avoid this, it is better to use regexp to strip out unnecessary information (like culture, publickkeytoken, etc.). So here's a replacement (needs to replace the above lines with this), a quiet rough one, but works in most cases:



typeName = Regex.Replace(typeName, @", Version=\d+.\d+.\d+.\d+", string.Empty);
typeName = Regex.Replace(typeName, @", Culture=\w+", string.Empty);
typeName = Regex.Replace(typeName, @", PublicKeyToken=\w+", string.Empty);

      

This regex does not cut the type assembly that is required for custom types. But for standard types it is mscorlib

, and it can be safely removed in most cases by adding another line:

typeName = Regex.Replace(typeName, @", mscorlib", string.Empty);

      

+3


source


I think memcache doesn't understand ProtoContract

, try to mark them asDataContract



0


source







All Articles