Powershell [Ref] A value that does not update the main object

I am running Powershell V2 on XP: This is part of a larger script and I notice some anomalies with how I can use referenced objects to update the "main" object. It started out as a way to avoid typing a complex property name every time - I could easily expand to a full name, but now the reason for this behavior is seriously eavesdropping on me.

[CmdletBinding()]
Param()
Function Breakhere {Write-Verbose " "}
Set-PSBreakpoint -Command Breakhere
$Data = @"
UserID
MyUser1
MyUser2
User3
"@
$Data = $Data|ConvertFrom-Csv

$Domains = "DomainA","DomainB"

$Props = "ParentContainer","AccountIsDisabled","employeeNumber"
$Connection = New-Object HashTable

ForEach ($Domain in $Domains)
{
    Write-Verbose "Current Domain: $Domain"
    # Add necessary headers to main data
    $text1 = "$($Domain)_ADObject"
    $text2 = "$($Domain)_Confidence"
    $Data = $Data |Select *,$text1
    $Data = $Data |Select *,$text2

    #Bind to each domain and save the connection contexts into a hashtable
    Write-Verbose "Binding to $Domain"
    $Connection.Add($Domain,(Connect-QADService -service $Domain))
}

ForEach ($User in $Data)
{
    ForEach ($Domain in $Domains)
    {
        $User."$($Domain)_ADObject" = Get-QADUser -Connection $Connection[$Domain] -SamAccountName $User.UserID -DontUseDefaultIncludedProperties -IncludedProperties $Props|Select $Props

        # Referencing the confidence parameter does not seem to work.
        $CF = [ref]$User."$($Domain)_Confidence"
        # Weirdly, this one does work.
        $AD = [ref]$User."$($Domain)_ADObject"

        If ($AD.Value)
        {
            $CF.Value = 1 
            Breakhere # Break here and allow for opportunity to inspect $user and $CF objects

            If ($AD.Value.AccountIsDisabled)
            {
                Write-Verbose "$Domain\$($User.UserID): Account Disabled"
                $CF.Value *= 0.8
            }
        }
        Else
        { 
            Write-Verbose "$Domain\$($User.UserID): No AD Object found"
            $CF.Value = 0
        }
    }
} #End ForEach $UserID

      

        
        
        
      

    

At a breakpoint, if I ask for $ User, I get something similar to the following:

UserID             : MyUser1
DomainA_ADObject   : @{ParentContainer=DomainA/Users; AccountIsDisabled=False; employeeNumber=123456}
DomainA_Confidence : 
DomainB_ADObject   : 
DomainB_Confidence : 

      

        
        
        
      

    

Things are good. Should I wish I could even use the ref AD AD object and update the DomainA_AD object:

$AD.Value.employeeNumber = 9999
$User

UserID             : MyUser1
DomainA_ADObject   : @{ParentContainer=DomainA/Users; AccountIsDisabled=False; employeeNumber=9999}
DomainA_Confidence : 
DomainB_ADObject   : 
DomainB_Confidence : 

      

        
        
        
      

    

However try this with $ CF ref and the same won't happen

$CF.Value = 2
$CF

Value
-----
2

$User

UserID             : MyUser1
DomainA_ADObject   : @{ParentContainer=DomainA/Users; AccountIsDisabled=False; employeeNumber=9999}
DomainA_Confidence :                         *<====== Expecting this to update!*
DomainB_ADObject   : 
DomainB_Confidence : 

      

        
        
        
      

    

Why the difference? Is there a way to query the [ref] object and see what it points to? I don't understand why one of them works and the other doesn't. They both seem to be tuned in the same way. Tried this in ISE and console, same behavior in both.