Apache Solr - Physical memory grows too fast on dataimport

I run the dataimport nightly command (127.0.0.1:8983/solr/dataimport?command=full-import)

After a few days I am looking at the solr control panel and the physical memory is over 4GB. If I remove the index (delete all files in the data directory) and then run dataimport, the physical memory drops to 1.1GB. Without removing it, physical memory grows very quickly.

Are additional parameters needed like clean = true? I want to run a cron task, which is essentially the same as deleting the data directory, and then run a full import to keep my physical memory as small as possible.

I am using the default cache settings for solr 4.0.

<query>

<maxBooleanClauses>1024</maxBooleanClauses>

<filterCache class="solr.FastLRUCache"
             size="512"
             initialSize="512"
             autowarmCount="0"/>


<queryResultCache class="solr.LRUCache"
                 size="512"
                 initialSize="512"
                 autowarmCount="0"/>


<documentCache class="solr.LRUCache"
               size="512"
               initialSize="512"
               autowarmCount="0"/>

<enableLazyFieldLoading>true</enableLazyFieldLoading>

<queryResultWindowSize>20</queryResultWindowSize>
<queryResultMaxDocsCached>200</queryResultMaxDocsCached>


<listener event="newSearcher" class="solr.QuerySenderListener">
  <arr name="queries">
    <!--
       <lst><str name="q">solr</str><str name="sort">price asc</str></lst>
       <lst><str name="q">rocks</str><str name="sort">weight asc</str></lst>
      -->
  </arr>
</listener>
<listener event="firstSearcher" class="solr.QuerySenderListener">
  <arr name="queries">
    <lst>
      <str name="q">static firstSearcher warming in solrconfig.xml</str>
    </lst>
  </arr>
</listener>

<useColdSearcher>false</useColdSearcher>

<maxWarmingSearchers>2</maxWarmingSearchers>

</query>