Spring Cache operations within the same transaction are not visible

Cache operations within the same transaction should be visible.

https://github.com/zhangheng0027/testCacheTransaction.git com.example.testcachetransaction.TestCacheTransactionApplicationTests#test02

@Log4j2
@CacheConfig(cacheNames = "cacheTransaction")
@Component
public class CacheTransaction {

    private static final AtomicInteger ai = new AtomicInteger(0);

    @Lazy
    @Resource
    CacheTransaction self;


    @Cacheable(key = "#s")
    public String cacheStr(String s) {
        return s + ai.incrementAndGet();
    }

    @Transactional
    public String cacheTransaction(String s) {
        String s1 = self.cacheStr(s);
        log.info("first cache: {}", s1);
        String s2 = self.cacheStr(s);
        log.info("second cache: {}", s2);

        return s2;
    }

}

    @Test
    public void test02() {
        Objects.requireNonNull(redisTemplate.getConnectionFactory()).getConnection().serverCommands().flushDb();
        cacheTransaction.cacheTransaction("abc");
        System.out.println(cacheTransaction.cacheStr("abc"));
    }

result:

2023-09-27T11:36:30.320+08:00  INFO 25076 --- [    Test worker] c.e.t.CacheTransaction                   : first cache: abc1
2023-09-27T11:36:30.323+08:00  INFO 25076 --- [    Test worker] c.e.t.CacheTransaction                   : second cache: abc2

Comment From: me0106

This is actually the operation of two data sources: jdbc and redis. Jdbc transactions do not guarantee repeatable reads of Redis data. If you need to take a snapshot of the cache, it should be saved to a local variable at the beginning of the transaction.

Comment From: zhangheng0027

This is actually the operation of two data sources: jdbc and redis. Jdbc transactions do not guarantee repeatable reads of Redis data. If you need to take a snapshot of the cache, it should be saved to a local variable at the beginning of the transaction.

But spring provides transaction support for cache, see org.springframework.cache.transaction.TransactionAwareCacheDecorator

Comment From: me0106

This is actually the operation of two data sources: jdbc and redis. Jdbc transactions do not guarantee repeatable reads of Redis data. If you need to take a snapshot of the cache, it should be saved to a local variable at the beginning of the transaction.

But spring provides transaction support for cache, see org.springframework.cache.transaction.TransactionAwareCacheDecorator

you are right. TransactionAwareCacheDecorator will delay caching operations until the transaction is completed. If there is no corresponding key in the cache,that's actually calling the real method. Before the transaction is completed All caches will not take effect. The reason for returning different values each time is that the 'cacheStr' method has side effects.

The cache is not visible because there is no cache at all.

Comment From: zhangheng0027

This is actually the operation of two data sources: jdbc and redis. Jdbc transactions do not guarantee repeatable reads of Redis data. If you need to take a snapshot of the cache, it should be saved to a local variable at the beginning of the transaction.

But spring provides transaction support for cache, see org.springframework.cache.transaction.TransactionAwareCacheDecorator

you are right. TransactionAwareCacheDecorator will delay caching operations until the transaction is completed. If there is no corresponding key in the cache,that's actually calling the real method. Before the transaction is completed All caches will not take effect. The reason for returning different values each time is that the 'cacheStr' method has side effects.

The cache is not visible because there is no cache at all.

Yes, it will be cached when the transaction is committed, which ensures isolation between different transactions, but the same transaction should not be isolated.

The following is my local solution, but I did not consider the propagation of transactions.

public class TransactionAwareCacheDecoratorIH extends TransactionAwareCacheDecorator {

    private static final ThreadLocal<Map<Object, TCache>> transactionCache = new ThreadLocal<>();

    public TransactionAwareCacheDecoratorIH(Cache targetCache) {
        super(targetCache);
    }

    protected static void init() {
        if (TransactionSynchronizationManager.isSynchronizationActive()) {
            Map<Object, TCache> map = transactionCache.get();
            if (map == null) {
                TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronization() {
                    @Override
                    public void afterCompletion(int status) {
                        transactionCache.remove();
                    }
                });
                map = new HashMap<>();
                transactionCache.set(map);
            }
        }
    }

    protected static TCache getTransactionCache(String key) {
        if (TransactionSynchronizationManager.isSynchronizationActive()) {
            init();
            return transactionCache.get().get(key);
        }
        return null;
    }

    protected void putInTransactionCache(Object key, String op, Object value) {
        if (TransactionSynchronizationManager.isSynchronizationActive()) {
            init();
            TCache tCache = new TCache(op);
            tCache.setValue(value);
            transactionCache.get().put(key, tCache);
        }
    }

    protected void clearTransactionCache() {
        if (TransactionSynchronizationManager.isSynchronizationActive()) {
            init();
            transactionCache.get().clear();
            transactionCache.get().put("clear::" + this.getName(), new TCache("clear"));
        }
    }


    @Override
    @Nullable
    public ValueWrapper get(Object key) {
        TCache byInTransactionCache = getTransactionCache(String.valueOf(key));
        if (null == byInTransactionCache)
            return super.get(key);
        if ("put".equals(byInTransactionCache.getType()))
            return new SimpleValueWrapper(byInTransactionCache.getValue());
        return null;
    }

    @Override
    public <T> T get(Object key, @Nullable Class<T> type) {
        ValueWrapper valueWrapper = get(key);
        if (null == valueWrapper)
            return null;
        return (T) valueWrapper.get();
    }

    @Override
    @Nullable
    @SneakyThrows
    public <T> T get(Object key, Callable<T> valueLoader) {
        ValueWrapper valueWrapper = get(key);
        if (null != valueWrapper)
            return (T) valueWrapper.get();
        return valueLoader.call();
    }

    @Override
    public void put(final Object key, @Nullable final Object value) {
        if (TransactionSynchronizationManager.isSynchronizationActive()) {
            putInTransactionCache(key, "put", value);
        }
        super.put(key, value);
    }

    @Override
    public void evict(final Object key) {
        if (TransactionSynchronizationManager.isSynchronizationActive()) {
            putInTransactionCache(key, "evict", null);
        }
        super.evict(key);
    }

    @Override
    public void clear() {
        if (TransactionSynchronizationManager.isSynchronizationActive()) {
            clearTransactionCache();
        }
        super.clear();
    }

    @Getter
    static public class TCache {

        /** 操作类型 **/
        String type;

        @Getter
        @Setter
        Object value;

        TCache(String type) {
            this.type = type;
        }

    }

}

Comment From: snicoll

@me0106 thanks for helping.

@zhangheng0027 unfortunately, you can't expect that level of support for TransactionAwareCacheDecorator, especially with two transactional resources.

Comment From: zhangheng0027

@me0106 thanks for helping.

@zhangheng0027 unfortunately, you can't expect that level of support for TransactionAwareCacheDecorator, especially with two transactional resources.

oh,no. This is one transaction, not two transactions

Comment From: zhangheng0027

@me0106 thanks for helping.

@zhangheng0027 unfortunately, you can't expect that level of support for TransactionAwareCacheDecorator, especially with two transactional resources.

I can provide another example to illustrate the inconsistency between cache and db

Comment From: zhangheng0027

@me0106 thanks for helping. @zhangheng0027 unfortunately, you can't expect that level of support for TransactionAwareCacheDecorator, especially with two transactional resources.

I can provide another example to illustrate the inconsistency between cache and db

https://github.com/zhangheng0027/testCacheTransaction.git com.example.testcachetransaction.service.impl.TUserServiceImplTest#test01

@Test
    public void test01() {

        TUser user = new TUser();
        user.setAge(15);
        user.setName("Tom");
        tUserService.save(user);

        // load to cache
        tUserService.getUser(user.getName());

        user.setAge(16);
        tUserService.updateAndGetUser(user);

    }

@Log4j2
@Service
@CacheConfig(cacheNames = "t_user")
public class TUserServiceImpl extends ServiceImpl<TUserMapper, TUser>
    implements TUserService {


    @Lazy
    @Resource
    TUserServiceImpl self;

    @Override
    @Cacheable(key = "#name")
    public TUser getUser(String name) {
        return lambdaQuery().eq(TUser::getName, name).last("limit 1").one();
    }

    @Override
    @CachePut
    public TUser updateUser(TUser user) {
        lambdaUpdate().set(TUser::getAge, user.getAge())
                .eq(TUser::getName, user.getName()).update();
        return user;
    }

    @Override
    @Transactional
    public TUser updateAndGetUser(TUser user) {
        TUser user1 = self.getUser(user.getName());
        log.info("update before user: {}", user1);

        self.updateUser(user);
        TUser user2 = self.getUser(user.getName());
        log.info("update after user: {}", user2);

        TUser one = lambdaQuery().eq(TUser::getName, user.getName()).last("limit 1").one();
        log.info("db user: {}", one);

        return user2;
    }


}

result

[    Test worker] c.e.t.service.impl.TUserServiceImpl : update before user: TUser [Hash = 85700, age=15, name=Tom]
[    Test worker] c.e.t.service.impl.TUserServiceImpl : update after user: TUser [Hash = 85700, age=15, name=Tom]
[    Test worker] c.e.t.service.impl.TUserServiceImpl : db user: TUser [Hash = 85731, age=16, name=Tom]

Comment From: zhangheng0027

@me0106 thanks for helping. @zhangheng0027 unfortunately, you can't expect that level of support for TransactionAwareCacheDecorator, especially with two transactional resources.

I can provide another example to illustrate the inconsistency between cache and db

https://github.com/zhangheng0027/testCacheTransaction.git com.example.testcachetransaction.service.impl.TUserServiceImplTest#test01

```java @Test public void test01() {

    TUser user = new TUser();
    user.setAge(15);
    user.setName("Tom");
    tUserService.save(user);

    // load to cache
    tUserService.getUser(user.getName());

    user.setAge(16);
    tUserService.updateAndGetUser(user);

}

```

```java @Log4j2 @Service @CacheConfig(cacheNames = "t_user") public class TUserServiceImpl extends ServiceImpl implements TUserService {

@Lazy
@Resource
TUserServiceImpl self;

@Override
@Cacheable(key = "#name")
public TUser getUser(String name) {
    return lambdaQuery().eq(TUser::getName, name).last("limit 1").one();
}

@Override
@CachePut
public TUser updateUser(TUser user) {
    lambdaUpdate().set(TUser::getAge, user.getAge())
            .eq(TUser::getName, user.getName()).update();
    return user;
}

@Override
@Transactional
public TUser updateAndGetUser(TUser user) {
    TUser user1 = self.getUser(user.getName());
    log.info("update before user: {}", user1);

    self.updateUser(user);
    TUser user2 = self.getUser(user.getName());
    log.info("update after user: {}", user2);

    TUser one = lambdaQuery().eq(TUser::getName, user.getName()).last("limit 1").one();
    log.info("db user: {}", one);

    return user2;
}

} ```

result

[ Test worker] c.e.t.service.impl.TUserServiceImpl : update before user: TUser [Hash = 85700, age=15, name=Tom] [ Test worker] c.e.t.service.impl.TUserServiceImpl : update after user: TUser [Hash = 85700, age=15, name=Tom] [ Test worker] c.e.t.service.impl.TUserServiceImpl : db user: TUser [Hash = 85731, age=16, name=Tom]

@snicoll Look at this example. This is a bug, open the issue please

Comment From: snicoll

As said before, you shouldn't expect changes during the transaction to be visible. TransactionAwareCacheDecorator is best efforts and will delay cache operations. If you need to get consistent cached data that is backed by a transactional data source, you need to use a second level cache.

Comment From: zhangheng0027

@snicoll The four basic principles of transactions, ACID, require transactions to ensure consistency. This bug violates the consistency of transactions.

Comment From: me0106

@snicoll The four basic principles of transactions, ACID, require transactions to ensure consistency. This bug violates the consistency of transactions.

ACID is only guaranteed in the same transaction within the same data source

Comment From: zhangheng0027

```java @Override @Transactional public TUser updateAndGetUser(TUser user) { TUser user1 = self.getUser(user.getName()); // output the value before the update log.info("update before user: {}", user1);

    // update database and cache
    self.updateUser(user);
    TUser user2 = self.getUser(user.getName());

    // output cached values after update
    log.info("update after user: {}", user2);

    TUser one = lambdaQuery().eq(TUser::getName, user.getName()).last("limit 1").one();
    // output db values after update
    log.info("db user: {}", one);

    return user2;
}

```

@snicoll The four basic principles of transactions, ACID, require transactions to ensure consistency. This bug violates the consistency of transactions.

ACID is only guaranteed in the same transaction within the same data source

Looking at my example, using @Transactional to start a transaction means that the entire method should be within the same transactional . You can predict log.info("update after user: {}", user2); result. It's different from our prediction.

The solution is to add cache within the current transaction, Record the modifications made to the transaction before this.

Comment From: me0106

@Transactional is only valid for Jdbc Transaction. SpringCache never guarantees the visibility of cache operations within a transaction, but simply delays cache operations until after the transaction is completed.

As described in the comments, SpringCache even allows you to directly modify the cache. :)